Almost State-of-the-art Text Generation library

Related tags

Text Data & NLPpythontensorflowtext-analysistext-generationpytorchlstm-neural-networkstext-gen
Overview

Ps: we are adding transformer model soon

Text Gen 🐐

Downloads python tensorflow PyPI

Almost State-of-the-art Text Generation library

Text gen is a python library that allow you build a custom text generation model with ease 😄 Something sweet built with Tensorflow and Pytorch(coming soon) - This is the brain of Rosalove ai (https://rosalove.xyz/)

How to use it

Install text-gen

pip install -U text-gen

import the library

from text_gen import ten_textgen as ttg

Load your data. your data must be in a text format.

Download the example data from the example folder

load data

data = 'rl.csv'
text = ttg.loaddata(data)

build our Model Architeture

pipeline = ttg.tentext(text)
seq_text = pipeline.sequence(padding_method = 'pre')
configg = pipeline.configmodel(seq_text, lstmlayer = 128, activation = 'softmax', dropout = 0.25)

train model

model_history = pipeline.fit(loss = 'categorical_crossentropy', optimizer = 'adam', batch = 300, metrics = 'accuracy', epochs = 500, verbose = 0, patience = 10)

generate text using the phrase

pipeline.predict('hello love', word_length = 200, segment = True)

plot loss and accuracy

pipeline.plot_loss_accuracy()

Hyper parameter optimization

Tune your model to know the best optimizer, activation method to use.

pipeline.hyper_params(epochs = 500)
pipeline.saveModel('model')

use a saved model for prediction

#the corpus is the train text file
ttg.load_model_predict(corpus = corpus, padding_method = 'pre', modelname = '../input/model2/model2textgen.h5', sample_text = 'yo yo', word_length = 100)

Give us a star ⭐ 🐉

If you want to contribute, take a look at the issues and the Futurework.md file

Contributors

Comments
  • use pipenv for managing dependencies

    use pipenv for managing dependencies

    Consider using (pipenv)[https://pypi.org/project/pipenv/] to pin your dependencies. This would allow contributors to easily reproduce the project without messing up the dependencies and its also good on the long run for maintainability

    opened by paularah 1
  • [Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

    [Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-PILLOW-1319443 | pillow:
    6.2.2 -> 8.3.2
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
  • Read on how to create a simple python library

    Read on how to create a simple python library

    https://towardsdatascience.com/how-to-build-your-first-python-package-6a00b02635c9

    https://medium.com/analytics-vidhya/how-to-create-a-python-library-7d5aea80cc3f

    opened by Emekaborisama 0
  • [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
tensorflow 1.14.0 requires protobuf, which is not installed.
tensorflow-serving-api 1.12.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires protobuf, which is not installed.
GPyOpt 1.2.6 requires GPy, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3180413 | wheel:
    0.30.0 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by Emekaborisama 0
  • [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
tensorflow 1.14.0 requires grpcio, which is not installed.
tensorflow 1.14.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires grpcio, which is not installed.
parameter-sherpa 1.0.6 requires pymongo, which is not installed.
parameter-sherpa 1.0.6 requires GPyOpt, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3092128 | wheel:
    0.30.0 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by Emekaborisama 0
  • [Snyk] Fix for 23 vulnerabilities

    [Snyk] Fix for 23 vulnerabilities

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
parameter-sherpa 1.0.6 requires scikit-learn, which is not installed.
GPy 1.10.0 requires paramz, which is not installed.
GPy 1.10.0 requires cython, which is not installed.
GPy 1.10.0 has requirement scipy<1.5.0,>=1.3.0, but you have scipy 1.2.3.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1055461 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1055462 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 509/1000
    Why? Has a fix available, CVSS 5.9 | Out-of-bounds Write
    SNYK-PYTHON-PILLOW-1059090 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1080635 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-PILLOW-1080654 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081494 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081501 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081502 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 654/1000
    Why? Has a fix available, CVSS 8.8 | Heap-based Buffer Overflow
    SNYK-PYTHON-PILLOW-1082329 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Insufficient Validation
    SNYK-PYTHON-PILLOW-1082750 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090584 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090586 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090587 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090588 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292150 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292151 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 566/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.6 | Buffer Overflow
    SNYK-PYTHON-PILLOW-1316216 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-Bounds
    SNYK-PYTHON-PILLOW-574573 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574574 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574575 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574576 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 469/1000
    Why? Has a fix available, CVSS 5.1 | Buffer Overflow
    SNYK-PYTHON-PILLOW-574577 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit low severity | 506/1000
    Why? Proof of Concept exploit, Has a fix available, CVSS 3.7 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-SCIKITLEARN-1079100 | scikit-learn:
    0.20.4 -> 0.24.2
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
Releases(v1.9.0)
Owner
Emeka boris ama
Machine Learning Engineer, Data Scientist, Youtuber and Advocacy
Emeka boris ama
GitHub Repository
The simple project to separate mixed voice (2 clean voices) to 2 separate voices.

Speech Separation The simple project to separate mixed voice (2 clean voices) to 2 separate voices. Result Example (Clisk to hear the voices): mix ||

vuthede 31 Oct 30, 2022
Mycroft Core, the Mycroft Artificial Intelligence platform.

Mycroft Mycroft is a hackable open source voice assistant. Table of Contents Getting Started Running Mycroft Using Mycroft Home Device and Account Man

Mycroft 6.1k Jan 09, 2023
A PyTorch Implementation of End-to-End Models for Speech-to-Text

speech Speech is an open-source package to build end-to-end models for automatic speech recognition. Sequence-to-sequence models with attention, Conne

Awni Hannun 647 Dec 25, 2022
This is the Alpha of Nutte language, she is not complete yet / Essa é a Alpha da Nutte language, não está completa ainda

nutte-language This is the Alpha of Nutte language, it is not complete yet / Essa é a Alpha da Nutte language, não está completa ainda My language was

catdochrome 2 Dec 18, 2021
Linking data between GBIF, Biodiverse, and Open Tree of Life

GBIF-biodiverse-OpenTree Linking data between GBIF, Biodiverse, and Open Tree of Life The python scripts will rely on opentree and Dendropy. To set up

2 Oct 03, 2022
BERT Attention Analysis

BERT Attention Analysis This repository contains code for What Does BERT Look At? An Analysis of BERT's Attention. It includes code for getting attent

Kevin Clark 401 Dec 11, 2022
Extracting Summary Knowledge Graphs from Long Documents

GraphSum This repo contains the data and code for the G2G model in the paper: Extracting Summary Knowledge Graphs from Long Documents. The other basel

Zeqiu (Ellen) Wu 10 Oct 21, 2022
基于pytorch+bert的中文事件抽取

pytorch_bert_event_extraction 基于pytorch+bert的中文事件抽取，主要思想是QA（问答）。 要预先下载好chinese-roberta-wwm-ext模型，并在运行时指定模型的位置。

西西嘛呦 31 Nov 30, 2022
Trains an OpenNMT PyTorch model and SentencePiece tokenizer.

Trains an OpenNMT PyTorch model and SentencePiece tokenizer. Designed for use with Argos Translate and LibreTranslate.

Argos Open Tech 61 Dec 13, 2022
this repository has datasets containing information of Uber pickups in NYC from April 2014 to September 2014 and January to June 2015. data Analysis , virtualization and some insights are gathered here

uber-pickups-analysis Data Source: https://www.kaggle.com/fivethirtyeight/uber-pickups-in-new-york-city Information about data set The dataset contain

1 Nov 02, 2021
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

Rasa Open Source Rasa is an open source machine learning framework to automate text-and voice-based conversations. With Rasa, you can build contextual

Rasa 15.3k Dec 30, 2022
100+ Chinese Word Vectors 上百种预训练中文词向量

Chinese Word Vectors 中文词向量 中文 This project provides 100+ Chinese Word Vectors (embeddings) trained with different representations (dense and sparse),

embedding 10.4k Jan 09, 2023
Simple Annotated implementation of GPT-NeoX in PyTorch

Simple Annotated implementation of GPT-NeoX in PyTorch This is a simpler implementation of GPT-NeoX in PyTorch. We have taken out several optimization

labml.ai 101 Dec 03, 2022
[NeurIPS 2021] Code for Learning Signal-Agnostic Manifolds of Neural Fields

Learning Signal-Agnostic Manifolds of Neural Fields This is the uncleaned code for the paper Learning Signal-Agnostic Manifolds of Neural Fields. The

60 Dec 12, 2022
Search with BERT vectors in Solr and Elasticsearch

Search with BERT vectors in Solr and Elasticsearch

Dmitry Kan 123 Dec 29, 2022
Search for documents in a domain through Google. The objective is to extract metadata

MetaFinder - Metadata search through Google _____ __ ___________ .__ .___ / \

Josué Encinar 85 Dec 16, 2022
[ICCV 2021] Instance-level Image Retrieval using Reranking Transformers

Instance-level Image Retrieval using Reranking Transformers Fuwen Tan, Jiangbo Yuan, Vicente Ordonez, ICCV 2021. Abstract Instance-level image retriev

UVA Computer Vision 86 Dec 28, 2022
Precision Medicine Knowledge Graph (PrimeKG)

PrimeKG Website | bioRxiv Paper | Harvard Dataverse Precision Medicine Knowledge Graph (PrimeKG) presents a holistic view of diseases. PrimeKG integra

Machine Learning for Medicine and Science @ Harvard 103 Dec 10, 2022
A Paper List for Speech Translation

Keyword: Speech Translation, Spoken Language Processing, Natural Language Processing

138 Dec 24, 2022
BookNLP, a natural language processing pipeline for books

BookNLP BookNLP is a natural language processing pipeline that scales to books and other long documents (in English), including: Part-of-speech taggin

654 Jan 02, 2023