Hello! The notebooks for evaluation in 2_Learn/2-5_eval-* all load the numpy object Q_star = np.load('qlearn_Q.npy'). However, in the Q-learning code, the learned Q-function is saved as np.save('L_qlearn_Q.npy', Q).
(from https://github.com/shengpu1126/RL-Set-Valued-Policy/blob/1f76cc789af96686dc34011ff28502f96d1565dc/mimic-sepsis/mimic_sepsis_rl/2_learn/2-3L_qlearn.py#L56 )

Should both qlearn_Q.npy and L_qlearn_Q.npy refer to the same object? I'm assuming not, because when I try to load Q_star = np.load('L_qlearn_Q.npy') in the eval notebook, the next step of the notebook throws an error due the assert statement in the soften-policies code (https://github.com/shengpu1126/RL-Set-Valued-Policy/blob/1f76cc789af96686dc34011ff28502f96d1565dc/mimic-sepsis/mimic_sepsis_rl/2_learn/2-5_eval-DR-WDR-err-Qstar.ipynb#L94)

It looks like something is missing. Am I misunderstanding something?

Thanks for the help!

There are two notebooks in 1_preprocess for generating trajD_tr.pkl, trajD_va.pkl, trajD_te.pkl. It is unclear which one between between 1_Z_reformat_data.ipynb vs 1_Z_reformat_data_discrete.ipynb is being used to generate those files.

It looks like 1_Z_reformat_data_discrete.ipynb is being used, but it'll be great if you can clarify this.