This project houses a collection of machine learning models including ANN, CNN, and a meta-model which is a combination of CNN and ANN. https://sketch2letter-82e3eb8c54bf.herokuapp.com/ is the web app that allows use to write letters by hand, and the pre-trained model will return the result. To train the model, ensure to set up your environment properly by following the steps outlined below.

To set up the necessary environment to train the models. We recommand using conda/miniconda. You will need to install several packages and python 3.10.12 You can install all the required packages using the following command:

pip install optuna numpy tensorflow tensorflow-datasets==4.0.1 scikit-learn tqdm matplotlib pydot graphviz

1. Load pre-trained ANN and CNN model.
2. Generate predictions from both ANN and CNN model.
3. stacks the predictions from the ANN and CNN models.
4. Creates, compiles, and trains a meta-model using the stacked predictions.
5. Saves the trained meta-model to the specified path.
6. Generates a confusion matrix and a classification report to evaluate the performance of the meta-model and saves the confusion matrix plot to a PNG file.

The working directory is FlaskApp

cd FlaskApp

To train the ANN model, use the following command in your terminal:

python ./ANN_src/bayesianTuning.py

To train the ANN model, use the following command in your terminal:

python ./CNN_src/cnn_model.py

To train the ANN model, use the following command in your terminal:

python ./META_model_src/meta_model.py

CNN is prone to making mistakes in the identification of (q,g), (i,l). The errors are mostly concentrated in partial recognition.

ANN is prone to making mistakes in the identification of (q,g), (i,l), but this model will return low confidense when the letter is ambiguous.

Meta is prone to making mistakes in the identification of (q,g), (i,l).

MIT