The visualization and results of SB3's DQN are in "homework answer/sb3_highway_dqn.html" The visualization has shown below.

The highway environment.

In this question, I selected merge environment, the running training and testing process are in "homework answer/sb3_highway_merge_dqn.html" The comments in baseline code could be found in "homework answer/sb3_merge_dqn.ipynb". The visualization of the testing result has shown below.

The merge environment.

ep_len_mean.

ep_rew_mean.

exploration rate.

time.