PersonaGPT

An open-domain conversational agent with many personalities

PersonaGPT is an open-domain conversational agent cpable of decoding personalized and controlled responses based on user input. It is built on the pretrained DialoGPT-medium model, following the GPT-2 architecture. PersonaGPT is fine-tuned on the Persona-Chat dataset, with added special tokens to better distinguish between conversational history and personality traits for dyadic conversations. Furthermore, some active learning was used to train the model to do controlled decoding based on certain "action codes" (e.g., talk about work, ask about music interests).

Personalized Dialog Generation

Also called persona modeling, personalized dialog generation takes as input a set of personality facts and dialog history to generate responses that are (ideally) sensible and diverse, yet consistent with past responses and the given persona profile.

Example

Controlled Response Generation

Another feature of PersonaGPT is generating responses that directs conversations toward certain topics of interests. The following is a self-play episode between 2 PersonaGPT agents, one interviews the other.

Interactive Environment

To interact with PersonaGPT, run interact.py --mode 0 to manually enter responses and see how the bot responds. You will first be prompted to enter a set of persona facts for your partner and then engage in a short conversation.

Use -turns or --num_turns to adjust the length of conversation (default = 8). To adjust the decoding parameters of PersonaGPT, use --help to bring up available settings to tune. For example, --top_k (default = 10) controls the number of candidate tokens at each decoding step, and --top_p controls the nucleus sampling. See how to generate text from Huggingface for nuances of the GPT decoding process.

To engage in controlled response generation, run interact.py --mode 1. At each turn, you will be prompted to select from a set of topics from which to steer the conversation toward. The output response by PersonaGPT may not always steer the conversation to the topic of interest, depending on the dialog history up to the current turn.

Training

Below are instructions to reconstruct the PersonaGPT from "scratch" (i.e., from pretrained DialoGPT or GPT-2, either of which are feasible as starting pre-trained models).

Requirements:

• Python 3.6+
• Pytorch (GPU preferred)
• transformers
• dotenv
• tqdm
• (optional) apex for fp16 training It is highly recommended that the pytorch and transformers packages are installed under a virtual environment.

After cloning this repository, follow the directions below to set up the training environment.

Instructions:

1. Go to the .env file and set the save_path to your desired local repository to store model, scheduler and optimizer checkpoints. Point data_path to the ~/data folder of the cloned repository. The .env file also contains the hyperparameter configurations:

epochs = 3
learn_rate = 5e-5
gradient_accumulation_steps = 64
batch_size = 1
weight_decay = 0.0
logging_steps = 10
save_steps = 250

Replace epochs, batch_size, gradient_accumulation_steps and learn_rate with the desired hyperparameters of choice. Please use batch_size = 1 and change gradient accumulation steps to adjust the training batch size. This current repo version does not support parallel batching at the moment (TODO).

2. Run preprocess_dataset.py to preprocess ~/data/train_both_original_no_cands.txt and ~/data/valid_both_original_no_cands.txt. The original .txt files are obtained from the ConvAI2 Challenge, which may no longer be available since the ConvAI3 challenge has taken place. The ConvAI2 challenge data uses the Persona-Chat dataset which is what is provided under the ~/data folder.

3. Run train.py to train the PersonaGPT model. Results (e.g., pretrain_loss, persona_loss, ctrl_loss) will be saved under [save_path]/samples/. Model checkpoints are saved under [save_path]/checkpoint/model.

Currently there are 2 training loops, pretrain() and train_loop(). pretrain() first trains model on the Persona-Chat dataset and saves the performance under pretrain_stats. train_loop() then fine-tunes the model on active learning data, which examples of using action codes (e.g., "talk about hobbies", "ask about pets") to do controlled response generation. The pretrained model can be used as as stand-alone dialog model for personalized dialog generation without fine-tuning on the actively learned actions.

• pretrain_loss: tracks the training loss on Persona-Chat dataset during pretrain().
• persona_loss: tracks the training loss on Persona-Chat during train_loop().
• ctrl_loss: tracks the training loss on actively learned action codes during train_loop().

Active Learning

Currently, there are 11 possible turn-level goals that can be used for controlled response generation.

These turn-level goals are handcrafted based on the personachat dataset to cover most of the conversational directions at the turn-level.

To actively learn new turn-level goals, use the convogym repo.

Evaluation

After training, an evaluation loop will run and print out a set of scores saved under eval_stats. Below is a comparison of PersonaGPT vs. other baselines on the Persona-Chat dataset using automatic evaluation metrics. Your results should look something like:

Cite Us

Our full paper is now up on arXiv.

@misc{tang2021persona,
      title={Persona Authentication through Generative Dialogue}, 
      author={Fengyi Tang and Lifan Zeng and Fei Wang and Jiayu Zhou},
      year={2021},
      eprint={2110.12949},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

References

1. Radford, Alec, et al. "Language models are unsupervised multitask learners." OpenAI Blog 1.8 (2019): 9.

2. Zhang, Yizhe, et al. "Dialogpt: Large-scale generative pre-training for conversational response generation." arXiv preprint arXiv:1911.00536 (2019).

3. Zhang, Saizheng, et al. "Personalizing dialogue agents: I have a dog, do you have pets too?." arXiv preprint arXiv:1801.07243 (2018).

4. Dinan et al., "The Second Conversational Intelligence Challenge (ConvAI2)." arXiv preprint arXiv:1902.00098 (2019).

5. Thomas Wolf et al. "Transfertransfo: A transfer learning approach for neural network based conversational agents." arXiv preprint328arXiv:1901.08149, 2019