A Neural Language Style Transfer framework to transfer natural language text smoothly between fine-grained language styles like formal/casual, active/passive, and many more. Created by Prithiviraj Damodaran. Open to pull requests and other forms of collaboration.

Overview

PyPI - License Visits Badge

Styleformer

A Neural Language Style Transfer framework to transfer natural language text smoothly between fine-grained language styles like formal/casual, active/passive, and many more.For instance, understand What makes text formal or casual/informal.

Table of contents

Usecases for Styleformer

Area 1: Data Augmentation

  • Augment training datasets with various fine-grained language styles.

Area 2: Post-processing

  • Apply style transfers to machine generated text.
  • e.g.
    • Refine a Summarised text to active voice + formal tone.
    • Refine a Translated text to more casual tone to reach younger audience.

Area 3: Controlled paraphrasing

  • Formal <=> Casual and Active <=> style transfers adds a notion of control over how we paraphrase when compared to free-form paraphrase where there is control or guarantee over the paraphrases.

Area 4: Assisted writing

  • Integrate this to any human writing interfaces like email clients, messaging tools or social media post authoring tools. Your creativity is your limit to te uses.
  • e.g.
    • Polish an email with business tone for professional uses.

Installation

pip install git+https://github.com/PrithivirajDamodaran/Styleformer.git

Quick Start

Casual to Formal (Available now !)

from styleformer import Styleformer
import torch
import warnings
warnings.filterwarnings("ignore")

'''
#uncomment for re-producability
def set_seed(seed):
  torch.manual_seed(seed)
  if torch.cuda.is_available():
    torch.cuda.manual_seed_all(seed)

set_seed(1234)
'''

# style = [0=Casual to Formal, 1=Formal to Casual, 2=Active to Passive, 3=Passive to Active etc..]
sf = Styleformer(style = 0) 

source_sentences = [
"I am quitting my job",
"Jimmy is on crack and can't trust him",
"What do guys do to show that they like a gal?",
"i loooooooooooooooooooooooove going to the movies.",
"That movie was fucking awesome",
"My mom is doing fine",
"That was funny LOL" , 
"It's piece of cake, we can do it",
"btw - ur avatar looks familiar",
"who gives a crap?",
"Howdy Lucy! been ages since we last met.",
"Dude, this car's dope!",
"She's my bestie from college",
"I kinda have a feeling that he has a crush on you.",
"OMG! It's finger-lickin' good.",
]   

for source_sentence in source_sentences:
    target_sentence = sf.transfer(source_sentence)
    print("-" *100)
    print("[Informal] ", source_sentence)
    print("-" *100)
    if target_sentence is not None:
        print("[Formal] ",target_sentence)
        print()
    else:
        print("No good quality transfers available !")
[Informal]  I am quitting my job
[Formal]  I will be stepping down from my job.
----------------------------------------------------------------------------------------------------
[Informal]  Jimmy is on crack and can't trust him
[Formal]  Jimmy is a crack addict I cannot trust him
----------------------------------------------------------------------------------------------------
[Informal]  What do guys do to show that they like a gal?
[Formal]  What do guys do to demonstrate their affinity for women?
----------------------------------------------------------------------------------------------------
[Informal]  i loooooooooooooooooooooooove going to the movies.
[Formal]  I really like to go to the movies.
----------------------------------------------------------------------------------------------------
[Informal]  That movie was fucking awesome
[Formal]  That movie was wonderful.
----------------------------------------------------------------------------------------------------
[Informal]  My mom is doing fine
[Formal]  My mother is doing well.
----------------------------------------------------------------------------------------------------
[Informal]  That was funny LOL
[Formal]  That was hilarious
----------------------------------------------------------------------------------------------------
[Informal]  It's piece of cake, we can do it
[Formal]  The whole process is simple and is possible.
----------------------------------------------------------------------------------------------------
[Informal]  btw - ur avatar looks familiar
[Formal]  Also, your avatar looks familiar.
----------------------------------------------------------------------------------------------------
[Informal]  who gives a crap?
[Formal]  Who cares?
----------------------------------------------------------------------------------------------------
[Informal]  Howdy Lucy! been ages since we last met.
[Formal]  Hello, Lucy It has been a long time since we last met.
----------------------------------------------------------------------------------------------------
[Informal]  Dude, this car's dope!
[Formal]  I find this car very appealing.
----------------------------------------------------------------------------------------------------
[Informal]  She's my bestie from college
[Formal]  She is my best friend from college.
----------------------------------------------------------------------------------------------------
[Informal]  I kinda have a feeling that he has a crush on you.
[Formal]  I have a feeling that he is attracted to you.
----------------------------------------------------------------------------------------------------
[Informal]  OMG! It's finger-lickin' good.
[Formal]  It is so good, it is delicious.
----------------------------------------------------------------------------------------------------

Knobs

# inference_on = [0=Regular model On CPU, 1= Regular model On GPU, 2=Quantized model On CPU]
target_sentence = sf.transfer(source_sentence, inference_on=0, quality_filter=0.95, max_candidates=5)

Models

Model Type Status
prithivida/informal_to_formal_styletransfer Seq2Seq Beta
prithivida/formal_to_informal_styletransfer Seq2Seq WIP
prithivida/active_to_passive_styletransfer Seq2Seq WIP
prithivida/passive_to_active_styletransfer Seq2Seq WIP
prithivida/positive_to_negative_styletransfer Seq2Seq WIP
prithivida/negative_to_positive_styletransfer Seq2Seq WIP

Dataset

  • TBD
  • Fined tuned on T5 on a Tesla T4 GPU and it took ~2 hours to train each of the above models with batch_size = 16 and epochs = 5.(Will share training args shortly)

Benchmark

  • TBD

References

Citation

  • TBD
Comments
  • added streamlit app

    added streamlit app

    Following points are covered in this PR:

    • Added Streamlit app. (CTF,FTC,ATP,PTA)
    • Fixed bug in PTA style transfer

    @PrithivirajDamodaran Attaching screenshot of streamlit app for reference. Let me know your suggestions

    app_screenshot

    opened by shashankdeshpande 6
  • Trimming long sentences

    Trimming long sentences

    Following the code snippet for a better understanding of the problem, I am facing.

    from styleformer import Styleformer
    import torch
    import warnings
    warnings.filterwarnings("ignore")
    
    # style = [0=Casual to Formal, 1=Formal to Casual, 2=Active to Passive, 3=Passive to Active etc..]
    sf = Styleformer(style = 0) 
    
    source_sentences = [
                       "Corruption in african countries hinders economic, political and social development. It is a major obstacle to economic growth, good governance and fundamental freedoms, such as freedom of speech or the right of citizens to hold governments accountable. In addition, corruption affects the lives of individuals, families and communities. The 10th global corruption barometer (gcb) program - in africa, shows that while many people in africa feel that corruption is on the rise in their country, many also feel confident that, as citizens, they can make a difference in the fight against corruption."
    ]
    
    for paragraph in source_sentences:
        # sentences = sent_tokenize(paragraph)
        sentences = paragraph.split('. ')
        for source_sentence in sentences:
            target_sentence = sf.transfer(source_sentence)
            print("-" *100)
            print("[Casual] ", source_sentence)
            print("-" *100)
            if target_sentence is not None:
                print("[Formal] ",target_sentence)
                print()
            else:
                print("No good quality transfers available !")
    

    Program Output


    [Casual] Corruption in african countries hinders economic, political and social development

    [Formal] In African countries, corruption affects economic, political, and social development.


    [Casual] It is a major obstacle to economic growth, good governance and fundamental freedoms, such as freedom of speech or the right of citizens to hold governments accountable

    [Formal] It's a major obstacle to economic growth, good governance, and fundamental freedoms, such as the freedom of speech or the right of citizens to


    [Casual] In addition, corruption affects the lives of individuals, families and communities

    [Formal] Additionally, corruption has a negative impact on individuals, families and communities.


    [Casual] The 10th global corruption barometer (gcb) program - in africa, shows that while many people in africa feel that corruption is on the rise in their country, many also feel confident that, as citizens, they can make a difference in the fight against corruption.

    [Formal] The tenth Global Corruptibility Barometer (GCB) program - in Africa - shows that while many people in Africa feel that corruption

    Please help to fix this for longer sentences. Thanks in advance!

    wontfix 
    opened by Nomiluks 4
  • OSError: Can't load config for 'prithivida/parrot_adequacy_on_BART'. Make sure that....

    OSError: Can't load config for 'prithivida/parrot_adequacy_on_BART'. Make sure that....

    Hi Prithviraj,

    Fantastic work you are doing.

    While testing your models, I intended to deploy the model wrapped in a flask app on EC2.

    Although the results work on Google Colab, I receive the following error on EC2 -

    OSError: Can't load config for 'prithivida/parrot_adequacy_on_BART'. Make sure that:
    
    - 'prithivida/parrot_adequacy_on_BART' is a correct model identifier listed on 'https://huggingface.co/models'
    
    - or 'prithivida/parrot_adequacy_on_BART' is the correct path to a directory containing a config.json file
    

    Can you guide me on how this can be resolved?

    Regards, Paritosh

    invalid 
    opened by katreparitosh 2
  • Issue with loading saved models

    Issue with loading saved models

    Hi, I'm trying to save and load the tokenizer and model. I use the following to save them:

    tokenizer = AutoTokenizer.from_pretrained("prithivida/informal_to_formal_styletransfer")
    tokenizer.save_pretrained('./data/style_tokenizer')
    model = AutoModelForSeq2SeqLM.from_pretrained("prithivida/informal_to_formal_styletransfer")
    model.save_pretrained('./data/style_model')
    

    But when I try to load them, from the local path, I get the following error:

    OSError: Can't load config for '../data/style_tokenizer'. Make sure that:
    
    - '../data/style_tokenizer' is a correct model identifier listed on 'https://huggingface.co/models'
    
    - or '../data/style_tokenizer' is the correct path to a directory containing a config.json file
    

    This somehow makes sense since saving the vectorizer, no config.json is being created.

    Any idea how can I save/load the tokenizer and model?

    opened by Naviden 1
  • Code to train the model

    Code to train the model

    Hey, Can you please share the code, where you train models? We have tasks similar to issues you solve but in other domains. It might be very helpful for us. Do you fine-tune only T5 or you make additional changes to T5 fine-tuning? Thanks

    question 
    opened by ivan-bulka 1
  • cant create a Styleformer(style=n)

    cant create a Styleformer(style=n)

    it keeps throing the same error(diffrent request id) OSError: There was a specific connection error when trying to load prithivida/informal_to_formal_styletransfer: <class 'requests.exceptions.HTTPError'> (Request ID: K9-6-Ks5uMEai7cOcQ3gC)

    opened by TalSchiff 0
  • Unable to create the styleformer instance

    Unable to create the styleformer instance

    OSError: prithivida/parrot_adequacy_on_BART is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'

    I'm using the latest version and seeing the following issue. I was wondering if anything has changed on the huggingface models front?

    opened by ks2002119 0
  • How to do inferencing using multiple GPU's for styleformer

    How to do inferencing using multiple GPU's for styleformer

    I am using this model to do inferencing on 1 million data point using A100 GPU's with 4 GPU. I am launching a inference.py code using Googles vertex-ai Container.

    How can I make inference code to utilise all 4 GPU's ? So that inferencing is super-fast.

    Here is the same code I use in inference.py:

    from styleformer import Styleformer
    import warnings
    warnings.filterwarnings("ignore")
    
    # style = [0=Casual to Formal, 1=Formal to Casual, 2=Active to Passive, 3=Passive to Active etc..]
    sf = Styleformer(style = 1) 
    import torch
    def set_seed(seed):
      torch.manual_seed(seed)
      if torch.cuda.is_available():
        torch.cuda.manual_seed_all(seed)
    
    set_seed(1212)
    
    source_sentences = [
    "I would love to meet attractive men in town",
    "Please leave the room now",
    "It is a delicious icecream",
    "I am not paying this kind of money for that nonsense",
    "He is on cocaine and he cannot be trusted with this",
    "He is a very nice man and has a charming personality",
    "Let us go out for dinner",
    "We went to Barcelona for the weekend. We have a lot of things to tell you.",
    ]   
    
    for source_sentence in source_sentences:
        # inference_on = [0=Regular model On CPU, 1= Regular model On GPU, 2=Quantized model On CPU]
        target_sentence = sf.transfer(source_sentence, inference_on=1, quality_filter=0.95, max_candidates=5)
        print("[Formal] ", source_sentence)
        if target_sentence is not None:
            print("[Casual] ",target_sentence)
        else:
            print("No good quality transfers available !")
        print("-" *100)     
    
    opened by pratikchhapolika 6
  • Sentiment Transfer

    Sentiment Transfer

    Love the library!

    Was hoping to do sentiment transfer but I see that has not yet been integrated. Any pointers towards off the shelf models that can do that?

    opened by JosephGatto 1
Releases(v1.0)
Owner
Prithivida
Applied NLP, XAI for NLP and Data Engineering
Prithivida
A Word Level Transformer layer based on PyTorch and đŸ¤— Transformers.

Transformer Embedder A Word Level Transformer layer based on PyTorch and đŸ¤— Transformers. How to use Install the library from PyPI: pip install transf

Riccardo Orlando 27 Nov 20, 2022
Smart discord chatbot integrated with Dialogflow

academic-NLP-chatbot Smart discord chatbot integrated with Dialogflow to interact with students naturally and manage different classes in a school. De

Tom Huynh 5 Oct 24, 2022
PyTorch implementation of the NIPS-17 paper "Poincaré Embeddings for Learning Hierarchical Representations"

Poincaré Embeddings for Learning Hierarchical Representations PyTorch implementation of Poincaré Embeddings for Learning Hierarchical Representations

Facebook Research 1.6k Dec 29, 2022
Tevatron is a simple and efficient toolkit for training and running dense retrievers with deep language models.

Tevatron Tevatron is a simple and efficient toolkit for training and running dense retrievers with deep language models. The toolkit has a modularized

texttron 193 Jan 04, 2023
A text file containing 479k English words for all your dictionary/word-based projects e.g: auto-completion / autosuggestion

List Of English Words A text file containing over 466k English words. While searching for a list of english words (for an auto-complete tutorial) I fo

dwyl 8.5k Jan 03, 2023
Textpipe: clean and extract metadata from text

textpipe: clean and extract metadata from text textpipe is a Python package for converting raw text in to clean, readable text and extracting metadata

Textpipe 298 Nov 21, 2022
JaQuAD: Japanese Question Answering Dataset

JaQuAD: Japanese Question Answering Dataset for Machine Reading Comprehension (2022, Skelter Labs)

SkelterLabs 84 Dec 27, 2022
Implementation of some unbalanced loss like focal_loss, dice_loss, DSC Loss, GHM Loss et.al

Implementation of some unbalanced loss for NLP task like focal_loss, dice_loss, DSC Loss, GHM Loss et.al Summary Here is a loss implementation reposit

121 Jan 01, 2023
TextFlint is a multilingual robustness evaluation platform for natural language processing tasks,

TextFlint is a multilingual robustness evaluation platform for natural language processing tasks, which unifies general text transformation, task-specific transformation, adversarial attack, sub-popu

TextFlint 587 Dec 20, 2022
A repo for open resources & information for people to succeed in PhD in CS & career in AI / NLP

A repo for open resources & information for people to succeed in PhD in CS & career in AI / NLP

420 Dec 28, 2022
Contains descriptions and code of the mini-projects developed in various programming languages

TexttoSpeechAndLanguageTranslator-project introduction A pleasant application where the client will be given buttons like play,reset and exit. The cli

Adarsh Reddy 1 Dec 22, 2021
BeautyNet is an AI powered model which can tell you whether you're beautiful or not.

BeautyNet BeautyNet is an AI powered model which can tell you whether you're beautiful or not. Download Dataset from here:https://www.kaggle.com/gpios

Ansh Gupta 0 May 06, 2022
Package for controllable summarization

summarizers summarizers is package for controllable summarization based CTRLsum. currently, we only supports English. It doesn't work in other languag

Hyunwoong Ko 72 Dec 07, 2022
Generate product descriptions, blogs, ads and more using GPT architecture with a single request to TextCortex API a.k.a Hemingwai

TextCortex - HemingwAI Generate product descriptions, blogs, ads and more using GPT architecture with a single request to TextCortex API a.k.a Hemingw

TextCortex AI 27 Nov 28, 2022
Index different CKAN entities in Solr, not just datasets

ckanext-sitesearch Index different CKAN entities in Solr, not just datasets Requirements This extension requires CKAN 2.9 or higher and Python 3 Featu

Open Knowledge Foundation 3 Dec 02, 2022
A sample project that exists for PyPUG's "Tutorial on Packaging and Distributing Projects"

A sample Python project A sample project that exists as an aid to the Python Packaging User Guide's Tutorial on Packaging and Distributing Projects. T

Python Packaging Authority 4.5k Dec 30, 2022
Predict an emoji that is associated with a text

Sentiment Analysis Sentiment analysis in computational linguistics is a general term for techniques that quantify sentiment or mood in a text. Can you

Tetsumichi(Telly) Umada 30 Sep 07, 2022
Original implementation of the pooling method introduced in "Speaker embeddings by modeling channel-wise correlations"

Speaker-Embeddings-Correlation-Pooling This is the original implementation of the pooling method introduced in "Speaker embeddings by modeling channel

Themos Stafylakis 10 Apr 30, 2022
This repository contains the code for EMNLP-2021 paper "Word-Level Coreference Resolution"

Word-Level Coreference Resolution This is a repository with the code to reproduce the experiments described in the paper of the same name, which was a

79 Dec 27, 2022
Utilizing RBERT model for KLUE Relation Extraction task

RBERT for Relation Extraction task for KLUE Project Description Relation Extraction task is one of the task of Korean Language Understanding Evaluatio

snoop2head 14 Nov 15, 2022