Almost State-of-the-art Text Generation library

Related tags

Text Data & NLPpythontensorflowtext-analysistext-generationpytorchlstm-neural-networkstext-gen
Overview

Ps: we are adding transformer model soon

Text Gen 🐐

Downloads python tensorflow PyPI

Almost State-of-the-art Text Generation library

Text gen is a python library that allow you build a custom text generation model with ease 😄 Something sweet built with Tensorflow and Pytorch(coming soon) - This is the brain of Rosalove ai (https://rosalove.xyz/)

How to use it

Install text-gen

pip install -U text-gen

import the library

from text_gen import ten_textgen as ttg

Load your data. your data must be in a text format.

Download the example data from the example folder

load data

data = 'rl.csv'
text = ttg.loaddata(data)

build our Model Architeture

pipeline = ttg.tentext(text)
seq_text = pipeline.sequence(padding_method = 'pre')
configg = pipeline.configmodel(seq_text, lstmlayer = 128, activation = 'softmax', dropout = 0.25)

train model

model_history = pipeline.fit(loss = 'categorical_crossentropy', optimizer = 'adam', batch = 300, metrics = 'accuracy', epochs = 500, verbose = 0, patience = 10)

generate text using the phrase

pipeline.predict('hello love', word_length = 200, segment = True)

plot loss and accuracy

pipeline.plot_loss_accuracy()

Hyper parameter optimization

Tune your model to know the best optimizer, activation method to use.

pipeline.hyper_params(epochs = 500)
pipeline.saveModel('model')

use a saved model for prediction

#the corpus is the train text file
ttg.load_model_predict(corpus = corpus, padding_method = 'pre', modelname = '../input/model2/model2textgen.h5', sample_text = 'yo yo', word_length = 100)

Give us a star ⭐ 🐉

If you want to contribute, take a look at the issues and the Futurework.md file

Contributors

Comments
  • use pipenv for managing dependencies

    use pipenv for managing dependencies

    Consider using (pipenv)[https://pypi.org/project/pipenv/] to pin your dependencies. This would allow contributors to easily reproduce the project without messing up the dependencies and its also good on the long run for maintainability

    opened by paularah 1
  • [Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

    [Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-PILLOW-1319443 | pillow:
    6.2.2 -> 8.3.2
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
  • Read on how to create a simple python library

    Read on how to create a simple python library

    https://towardsdatascience.com/how-to-build-your-first-python-package-6a00b02635c9

    https://medium.com/analytics-vidhya/how-to-create-a-python-library-7d5aea80cc3f

    opened by Emekaborisama 0
  • [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
tensorflow 1.14.0 requires protobuf, which is not installed.
tensorflow-serving-api 1.12.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires protobuf, which is not installed.
GPyOpt 1.2.6 requires GPy, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3180413 | wheel:
    0.30.0 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by Emekaborisama 0
  • [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
tensorflow 1.14.0 requires grpcio, which is not installed.
tensorflow 1.14.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires grpcio, which is not installed.
parameter-sherpa 1.0.6 requires pymongo, which is not installed.
parameter-sherpa 1.0.6 requires GPyOpt, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3092128 | wheel:
    0.30.0 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by Emekaborisama 0
  • [Snyk] Fix for 23 vulnerabilities

    [Snyk] Fix for 23 vulnerabilities

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
parameter-sherpa 1.0.6 requires scikit-learn, which is not installed.
GPy 1.10.0 requires paramz, which is not installed.
GPy 1.10.0 requires cython, which is not installed.
GPy 1.10.0 has requirement scipy<1.5.0,>=1.3.0, but you have scipy 1.2.3.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1055461 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1055462 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 509/1000
    Why? Has a fix available, CVSS 5.9 | Out-of-bounds Write
    SNYK-PYTHON-PILLOW-1059090 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1080635 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-PILLOW-1080654 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081494 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081501 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081502 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 654/1000
    Why? Has a fix available, CVSS 8.8 | Heap-based Buffer Overflow
    SNYK-PYTHON-PILLOW-1082329 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Insufficient Validation
    SNYK-PYTHON-PILLOW-1082750 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090584 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090586 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090587 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090588 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292150 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292151 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 566/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.6 | Buffer Overflow
    SNYK-PYTHON-PILLOW-1316216 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-Bounds
    SNYK-PYTHON-PILLOW-574573 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574574 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574575 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574576 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 469/1000
    Why? Has a fix available, CVSS 5.1 | Buffer Overflow
    SNYK-PYTHON-PILLOW-574577 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit low severity | 506/1000
    Why? Proof of Concept exploit, Has a fix available, CVSS 3.7 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-SCIKITLEARN-1079100 | scikit-learn:
    0.20.4 -> 0.24.2
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
Releases(v1.9.0)
Owner
Emeka boris ama
Machine Learning Engineer, Data Scientist, Youtuber and Advocacy
Emeka boris ama
GitHub Repository
A Python/Pytorch app for easily synthesising human voices

Voice Cloning App A Python/Pytorch app for easily synthesising human voices Documentation Discord Server Video guide Voice Sharing Hub FAQ's System Re

Ben Andrew 840 Jan 04, 2023
运小筹公众号是致力于分享运筹优化(LP、MIP、NLP、随机规划、鲁棒优化)、凸优化、强化学习等研究领域的内容以及涉及到的算法的代码实现。

OlittleRer 运小筹公众号是致力于分享运筹优化(LP、MIP、NLP、随机规划、鲁棒优化)、凸优化、强化学习等研究领域的内容以及涉及到的算法的代码实现。编程语言和工具包括Java、Python、Matlab、CPLEX、Gurobi、SCIP 等。 关注我们: 运筹小公众号 有问题可以直接在

运小筹 151 Dec 30, 2022
Use Google's BERT for named entity recognition （CoNLL-2003 as the dataset）.

For better performance, you can try NLPGNN, see NLPGNN for more details. BERT-NER Version 2 Use Google's BERT for named entity recognition （CoNLL-2003

Kaiyinzhou 1.2k Dec 26, 2022
This repository contains the codes for LipGAN. LipGAN was published as a part of the paper titled "Towards Automatic Face-to-Face Translation".

LipGAN Generate realistic talking faces for any human speech and face identity. [Paper] | [Project Page] | [Demonstration Video] Important Update: A n

Rudrabha Mukhopadhyay 438 Dec 31, 2022
SDL: Synthetic Document Layout dataset

SDL is the project that synthesizes document images. It facilitates multiple-level labeling on document images and can generate in multiple languages.

Sơn Nguyễn 0 Oct 07, 2021
Examples of using sparse attention, as in "Generating Long Sequences with Sparse Transformers"

Status: Archive (code is provided as-is, no updates expected) Update August 2020: For an example repository that achieves state-of-the-art modeling pe

OpenAI 1.3k Dec 28, 2022
Spokestack is a library that allows a user to easily incorporate a voice interface into any Python application with a focus on embedded systems.

Welcome to Spokestack Python! This library is intended for developing voice interfaces in Python. This can include anything from Raspberry Pi applicat

Spokestack 133 Sep 20, 2022
Scene Text Retrieval via Joint Text Detection and Similarity Learning

This is the code of "Scene Text Retrieval via Joint Text Detection and Similarity Learning". For more details, please refer to our CVPR2021 paper.

79 Nov 29, 2022
Wind Speed Prediction using LSTMs in PyTorch

Implementation of Deep-Forecast using PyTorch Deep Forecast: Deep Learning-based Spatio-Temporal Forecasting Adapted from original implementation Setu

Onur Kaplan 151 Dec 14, 2022
A BERT-based reverse-dictionary of Korean proverbs

Wisdomify A BERT-based reverse-dictionary of Korean proverbs. 김유빈 : 모델링 / 데이터 수집 / 프로젝트 설계 / back-end 김종윤 : 데이터 수집 / 프로젝트 설계 / front-end Quick Start C

Eu-Bin KIM 94 Dec 08, 2022
Script to generate VAD dataset used in Asteroid recipe

About the dataset LibriVAD is an open source dataset for voice activity detection in noisy environments. It is derived from LibriSpeech signals (clean

11 Sep 15, 2022
open-information-extraction-system, build open-knowledge-graph(SPO, subject-predicate-object) by pyltp(version==3.4.0)

中文开放信息抽取系统, open-information-extraction-system, build open-knowledge-graph(SPO, subject-predicate-object) by pyltp(version==3.4.0)

7 Nov 02, 2022
Calibre recipe to convert latest issue of Analyse & Kritik into an ebook

Calibre Recipe für "Analyse & Kritik" Dies ist ein "Recipe" für die Konvertierung der aktuellen Ausgabe der Zeitung Analyse & Kritik in ein Ebook. Es

Henning 3 Jan 04, 2022
NLP project that works with news (NER, context generation, news trend analytics)

СоАвтор СоАвтор – платформа и открытый набор инструментов для редакций и журналистов-фрилансеров, который призван сделать процесс создания контента ма

38 Jan 04, 2023
NLP techniques such as named entity recognition, sentiment analysis, topic modeling, text classification with Python to predict sentiment and rating of drug from user reviews.

This file contains the following documents sumbited for Baruch CIS9665 group 9 fall 2021. 1. Dataset: drug_reviews.csv 2. python codes for text classi

Aarif Munwar Jahan 2 Jan 04, 2023
A flask application to predict the speech emotion of any .wav file.

This is a speech emotion recognition app. It will allow you to train a modular MLP model with the RAVDESS dataset, and then use that model with a flask application to predict the speech emotion of an

Aryan Vijaywargia 2 Dec 15, 2021
Nmt - TensorFlow Neural Machine Translation Tutorial

Neural Machine Translation (seq2seq) Tutorial Authors: Thang Luong, Eugene Brevdo, Rui Zhao (Google Research Blogpost, Github) This version of the tut

6.1k Dec 29, 2022
A multi-voice TTS system trained with an emphasis on quality

TorToiSe Tortoise is a text-to-speech program built with the following priorities: Strong multi-voice capabilities. Highly realistic prosody and inton

James Betker 2.1k Jan 01, 2023
Generate text line images for training deep learning OCR model (e.g. CRNN)

Generate text line images for training deep learning OCR model (e.g. CRNN)

532 Jan 06, 2023
UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language

UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language This repository contains UA-GEC data and an accompanying Python lib

Grammarly 227 Jan 02, 2023