We use the label vocabulary from the Gector available here. For our project, we do not need the @@UNKNOWN@@ and @@PADDING@@ labels since they are added automatically because of Gector's implementation of the Allennlp library.

Weighted Sampler

The label vocabulary contains 5,000 labels for 9 categories; keep, delete, replace, append, merge, transform case, transform verb, transform agreement and transform split. The number of labels in each category is shown in the graph below.

Randomly sampling the action label will be inefficient for the following reasons:

1. The probability of a random correct guess is really low because of the high action space.
2. Some labels like keep, replace and append occur more often than other labels.
3. Randomly deleting tokens will lose the information from the original sentence.
4. Randomly appending tokens will change the context of the original sentence.

Therefore, we created a weighted sampler that generates some action labels more often than others.

Test

For the test, we used the following label weights in the WeightedSampler:

{
    "$KEEP": 3.0,
    "$MERGE": 0.1,
    "$DELETE": 0.3,
    "$APPEND": 0.5,
    "$REPLACE": 1.0, 
    "$TRANSFORM_SPLIT": 0.2,
    "$TRANSFORM_CASE": 1.0,
    "$TRANSFORM_VERB": 1.0,
    "$TRANSFORM_AGREEMENT": 1.0,
}

Using the above weights, we sampled different amounts of samples.

The sample above depicts the scenario where we randomly sample the action labels for a sentence with 10 tokens. Because of the high weight on the keep label, most of the random labels are in this category.

The sample above depicts the likelihood of sampling different action labels during the training stage.

from drl-gec.