This project allows to take the Twitter Customer Support and format it in the Persona Chat format. This is helpful to adapt the model described in this paper into an task oriented version.

• Getting Started
• Usage
• Contribute
  • Contributors
• Show you Support
• License

Click here and extract the zip to your preferred dirctory.

The first step is to install pipenv. Go to the project directory and run: On mac: You can use homebrew:

brew install pipenv

or pip:

pip install pipenv

On Linux:

sudo apt install software-properties-common python-software-properties
sudo add-apt-repository ppa:pypa/ppa
sudo apt update
sudo apt install pipenv

On the project directory run:

pip install -r requirements.txt

To run the project:

python cli.py [module_name] [options]

This module allows you to retrieve some metadata about the Twitter Customer Support to use it run:

python cli.py getMetadata

This module allows you to preprocess the Twitter Customer Support. Here are the options you can use:

• --emojis: Boolean, if True, removes all emojis from the dataset (default: True)
• --emoticons: Boolean, if True, removes all emoticons from the dataset (default: True)
• --urls: Boolean, if True, tags urls as '(URL)' from the dataset (default: True)
• --html_tags: Boolean, if True, removes all html tags (default: True)
• --acronyms: Boolean, if True, converts acronyms to their meaning. E.g.: SMH -> So much Hate (default: True)
• --spelling: Boolean, if True, spellchecks the dataset (default: False)
• --usernames: Boolean, if True, tags usernames (default: False)

To run:

python cli.py preprocess [options]

This modules allows you to format the (preprocessed or not) dataset. The options are:

• --brand: String, represents the name of a brand, only uses the interactions with a specific brand. If none, uses the whole dataset (default: None)
• --limit: Integer, only uses a limited amount of conversations. If -1 uses the whole dataset (default: -1)

If you have any ideas, just open an issue and tell us what you think!

If you'd like to contribute, please fork the repository and make changes as you'd like. Pull requests are warmly welcome.

This project is MIT licensed.