over-documented over-explained chatbot using original transformer as sequence-to-sequence model

My research interest is in incorporating the the latest research in natural language processing and reinforcement learning into palpable human interaction such as fully neural network based conversational dialogue.

chloe is an end to end neural network chatbot written in PyTorch based on the transformer. Accomplishing goals through conversation is a task we can all relate to, thus chatbots are an ideal agent through which to connect new research to our current understanding of AI and demonstrate new research. chloe is built from modules that are thoroughly explained and demonstrated using jupyter notebooks and toy examples that build from these basics.

• Python 3.6 to 3.8
• torch==1.7.1 (PyTorch)
• torchtext==0.4.0
• nltk==3.4.5 (Natural Language Toolkit)

• START_HERE introduction, summon and meet chloe
• Trainer Tokenization in Sequnce to Sequence learning, intuitive explaination of Loss Functions using a toy example of cross entropy
• Building Blocks of Neural Networks Word Vector Embeddings, Positional Encoding, Neural Networks, Attention, Masking
• The Network Architecture Assembling the building blocks into an encoder and decoder
• Talk How chloe chooses which words to say in a sentence and some basic probability

if you already have python 3.6 and virtual environments, create a python 3.6 virtual environment, here i used env36 for python3.6 but you can use anything

python3 -m venv env or $ python3.6 -m venv env or virtualenv --python=/usr/bin/python3.6 env

if python3.6 is your default version, then when you type python into your terminal then it should say python version 3.6.x, and for you making the virtual environment is as simple as

$ python -m venv env

otherwise it is simple to get python3.6 and virtual environments

how to install Python 3.6 on ubuntu

install virtual environment then

how to specify the Python executable you want to use

sudo add-apt-repository ppa:jonathonf/python-3.6

sudo apt-get update

sudo apt-get install python3.6

virtualenv --python=/usr/bin/python3.6 env36

When you want to run the code activate the virtual environment inside the same folder as your environment env using

$ source env/bin/activate

install dependencies

$ pip3 install -r requirements.txt

even with virtual environments, some troubleshoot might be needed

with enough google searches you can find an answer for almost any problem

save new dependences to requirements

$ pip3 freeze > requirements.txt

You can deactivate the virtual environment using the following command in your terminal:

$ deactivate

if you get a ImportError: No module named while at the same time in your Terminal you get

pip3 install import-ipynb

Requirement already satisfied: import-ipynb in /path/to/env/lib/python3.6/site-packages (0.1.3)

This can be fixed by providing your python interpreter with the path-to-your-module,the path

import sys

sys.path.append('/path/to/env/lib/python3.6/site-packages')

you can find this path listed here

$ python3 -m site

• Python tutorials
• jupyter notebooks
• 3Blue1Brown's Essence of linear algebra
• 3Blue1Brown's Neural Networks
• PyTorch tutorial