Build multi-turn Chatbot GPT-2

The following code trains a multi-turn chatbot model based on the DailyDialog multi-turn dataset and pre-trained GPT-2 model.

Multi-turn GPT-2 Dialogue Results

"Q" represents user (your inputs) message, and "A" represents the chatbot's response.

Q: Have you ever been to France?
A: Oh, yes, I'Ve been there. I went there last summer.
Q: How was there?
A: It was great!
Q: Which spot was your best?
A: I like the place, don't you?
Q: What your favourite place in France?
A: It's certainly of interest to me.
Q: Best place in France?
A: Yes, it's very famous.
Q: Famous places in France?
A: Yes, like Eiffel Tower, Eiffel Tower and so on.
Q: What about Louvre?
A: It's very famous, but not very modern.
Q: National French cuisine
A: That's very interesting.
Q: What about National French cuisine
A: It's got ingredients.


Q: Did you see her yesterday?
A: Yes, I did. I saw her with a pretty girl.
Q: Oh where did you see her?                       
A: She was in a cafe. She told me she was going to a concert.
Q: I see. I've tried to contact her by phone, but she never received my call.
A: I'm sure she was in New York. She said she'd be in New York at the latest.
Q: Really? Thank you for your information. I will contact her today. Thanks.

Quick Start 🚀

python3 src/run/train.py --config config/project.yaml --mode train

Supported Devices

• CPU, GPU, multi-GPU (DDP), MPS (for Mac and torch>=1.12.0)

Supported Models

• Pre-trained GPT-2 from Hugging Face: "gpt2" from Hugging Face.

Supported Tokenizer

• Pre-trained GPT-2 tokenizer from Hugging Face.

Base Dataset 📚

• DailyDialog multi-turn dataset.
• If you want to use your custom data, you have to set train/validation/test data paths in the config/project.yaml. Also, you have to implement your custom tokenizer, data loading parts in the src/trainer/build.py.

Tutorials & Documentations 🔍

Please follow the steps below to train a multi-turn chatbot model.

1. Getting Started
2. Data Preparation
3. Training
4. ETC
   • Evaluation
   • Live Demo
     
     

Training Results 📣

Results of GPT-2-based Multi-turn Chatbot Model

• BLEU Score History
  
  
  

• NIST Score History
  
  
  

• Test Set Scores
  The results of applying the model, which achieved the highest metric on the validation set, to the test set.

  • BLEU-2: 0.4052
  • BLEU-4: 0.2268
  • NIST-2: 5.1797
  • NIST-4: 5.5162