TKT is an extension of EduKTM, which has more advanced features making the training, evaluating and deployment faster and easier.

See Examples and Scripts for quick start. The beginner is advised to first get familiar with EduKTM.

With EduData, we test the models performance, the AUC result is listed as follows:

The f1 scores are listed as follows:

The information of the benchmark datasets can be found in EduData docs.

In addition, all models are trained 20 epochs with batch_size=16, where the best result is reported. We use adam with learning_rate=1e-3. We also apply bucketing to accelerate the training speed. Moreover, each sample length is limited to 200. The hyper-parameters are listed as follows:

The number after - in the first row indicates the knowledge units number in the dataset. The datasets we used can be either found in basedata-ktbd or be downloaded by:

pip install EduData
edudata download ktbd

• DKT: hidden_num is usually set to the nearest hundred number to the ku_num
• EmbedDKT: latent_dim is usually set to a value litter than or equal to \sqrt(hidden_num * ku_num)
• DKVMN: key_embedding_dim = key_memory_state_dim and value_embedding_dim = value_memory_state_dim

Some interfaces of pytorch may change with version changing, such as

import torch
torch.nn.functional.one_hot

which may caused some errors like:

AttributeError: module 'torch.nn.functional' has no attribute 'one_hot'

Except that, there is a known bug Segmentation fault: 11:

Segmentation fault: 11

Stack trace:
  [bt] (0) /usr/local/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2e6b160) [0x7f3e4b5b6160]
  [bt] (1) /lib64/libc.so.6(+0x36340) [0x7f3ec3c89340]
  [bt] (2) /usr/local/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x40a5760) [0x7f3dc265c760]
  [bt] (3) /usr/local/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x40a35c5) [0x7f3dc265a5c5]
  [bt] (4) /lib64/libstdc++.so.6(+0x5cb19) [0x7f3eb807db19]
  [bt] (5) /lib64/libc.so.6(+0x39c29) [0x7f3ec3c8cc29]
  [bt] (6) /lib64/libc.so.6(+0x39c77) [0x7f3ec3c8cc77]
  [bt] (7) /lib64/libc.so.6(__libc_start_main+0xfc) [0x7f3ec3c7549c]
  [bt] (8) python3() [0x41da20]

However, the mentioned-above bug does not affect the train and evaluation.

PS. if you think those problems are so easy to solve, please do not hesitate to contact us :-).

1. First get the repo in your computer by git or any way you like.
2. Suppose you create the project under your own home directory, then you can use use
   1. pip install -e . to install the package, or
   2. export PYTHONPATH=$PYTHONPATH:~/TKT

In TKT, all sequence is store in json format, such as:

[[419, 1], [419, 1], [419, 1], [665, 0], [665, 0]]

Each item in the sequence represent one interaction. The first element of the item is the exercise id and the second one indicates whether the learner correctly answer the exercise, 0 for wrongly while 1 for correctly
One line, one json record, which is corresponded to a learner's interaction sequence.

A demo loading program is presented as follows:

import json
from tqdm import tqdm

def extract(data_src):
    responses = []
    step = 200
    with open(data_src) as f:
        for line in tqdm(f, "reading data from %s" % data_src):
            data = json.loads(line)
            for i in range(0, len(data), step):
                if len(data[i: i + step]) < 2:
                    continue
                responses.append(data[i: i + step])

    return responses

The above program can be found in TKT/TKT/shared/etl.py.

To deal with the issue that the dataset is store in tl format:

5
419,419,419,665,665
1,1,1,0,0

Refer to Edudata Documentation.

All command to invoke the model has the same cli canonical form:

python Model.py $mode $parameters1 $parameters2 ...

There are a lot of models that implements different knowledge tracing models in different frameworks, the following are the url of those implemented by python (the stared is the authors version):

• DKT [tensorflow]

• DKT+ [tensorflow*]

• DKVMN [mxnet*]

• KTM [libfm]

• EKT[pytorch*]

There are some datasets which are suitable for this task, you can refer to BaseData ktbd doc for these datasets