kitsamho / fine_tuning_bert Goto Github PK

In this exercise, you will have to create a BERT model and then train it on the CoLA dataset which has a list of sentences that are either grammatically correct or incorrect. The data loading and model training, testing logic are already included in your code. You will need to fetch the data, and use a pretrained BERT model to solve a classification task.

In this workspace you have GPU to help train the model but it is best practice to DISABLE it while writing code and only ENABLE it when you are training.

Here are the steps you need to do to complete this exercise:

• Data has already been downloaded from here into the cola_public directory.
• Create a tokenizer for the BERT Model
• Create the BERT Model and the optimizer for the model
• Save all your work and then ENABLE the GPU
• Run the Package Installations.
• Run the file to make sure that the model is training properly.
• If it works, remember to DISABLE the GPU before moving to the next page.