Add the link to the IEEE website and authors' information.

Currently, in the latest code from Kueitso, some functions and symbols have unclear purposes. We need to check the paper and refine them (either removing them if they are not used or adding references to the paper in the comments).

Functions

• infer_tau2
• run_uniformA

Symbols

If the symbols are not defined in the paper, we can change the name to something more descriptive.

• ball_control_k1 and ball_control_k2
• action_control_constant_mu1

Related PR

After this issue is solved, we can keep moving forward on #14.

In BaseballGame, the state only consists of two integers, which is strike and ball count. It's better to represent them using a tuple (or even better, a namedtuple) instead of a mutable list (which doesn't show exactly what is in a state).

There are only three experiments left in the latest version of the paper, add those three and remove the rest.

In this task, only make sure the new experiments work with the baseball environment, backward compatibility with the existing environments will be another task if needed.

• Apply experiment 1 on the baseball environment
• #52
• #53

The final version of the paper only contains the baseball environment, we need to include a description of how the previous three environments work.

Check the pre-commit project.

Currently, there are three environments in this repo. However, the final version of the paper demonstrates BSI and BSI-PT through a new baseball game environment. This task is to include this environment and ensure that we can conduct the experiments in different environments in a consistent manner.

• #43
• #44
• #45
• #46

Since KL divergence is not used in the paper anymore, remove the section related to this (in each environment's run.py and plot.py ). Also, once the KL divergence code is removed, remove the scipy dependency that is used to calculate KL divergence.

Currently, the test scripts running in the CI are under scripts/, and should move to a new folder. (probably called tests/)

Not really an issue, but maybe consider adding https://dangitgit.com/en into our documentation. There are quite some useful solutions to Git problems.

Currently it's named ci, not very descriptive.

Replace the current link to the publishers' website.

Material for MkDocs seems to be a nice option, check this link: https://squidfunk.github.io/mkdocs-material/publishing-your-site/

Once this is set up, the wiki page can be removed.

run.py should work with the following agent-opponent combinations.

• BSI-PT vs PhiOpponent
• BSI-PT vs PhiNoiseOpponent
• OKR vs PhiOpponent
• OKR vs PhiNoiseOpponent
• Deep BPR+ vs PhiOpponent
• Deep BPR+ vs PhiNoiseOpponent
• BPR+ vs PhiOpponent
• BPR+ vs PhiNoiseOpponent

Currently, even if only the documentation is being modified, the tests will still be executed.

Add mkdocs documentation in the developer section.

• how to run mkdocs locally

To pass the mypy type check, I initialized performance_model as None (see the examples below).

https://github.com/jerry871002/bayesian-strategy-inference/blob/e71229871278286cf72590ea3bff5508886092fa/src/grid_world/agent.py#L29

https://github.com/jerry871002/bayesian-strategy-inference/blob/e71229871278286cf72590ea3bff5508886092fa/src/navigation_game/agent.py#L27

https://github.com/jerry871002/bayesian-strategy-inference/blob/e71229871278286cf72590ea3bff5508886092fa/src/soccer_game/agent.py#L29

But I didn't add any check before using it to ensure the value is properly set, e.g. is it properly set to an array-like value and not None. Need to fix this in the future.

https://github.com/Saransh-cpp/OCRed/blob/main/mkdocs.yml
https://github.com/LuxDL/Lux.jl/blob/main/docs/mkdocs.yml

https://github.com/jerry871002/bayesian-strategy-inference/actions/runs/5841852816/job/15842391174

+ python run.py baseball bpr-okr -n 5 --new-phi-opponent -q 3
----- (bpr-okr agent, New Phi opponent) q = 3 -----
Traceback (most recent call last):
  File "/home/runner/work/bayesian-strategy-inference/bayesian-strategy-inference/src/run.py", line 195, in <module>
    run(args)
  File "/home/runner/work/bayesian-strategy-inference/bayesian-strategy-inference/src/run.py", line 49, in run
    run_bpr_okr(args)
  File "/home/runner/work/bayesian-strategy-inference/bayesian-strategy-inference/src/baseball_game/run.py", line 355, in run_bpr_okr
    policy_preds.append(step_5_policy_preds[i])
IndexError: list index out of range

https://github.com/jerry871002/bayesian-strategy-inference/actions/runs/5857902074/job/15880785779?pr=56

+ python run.py baseball bsi-pt -n 5 -e 2

----- (bsi-pt agent) Test random switch opponent -----
Traceback (most recent call last):
  File "run.py", line 195, in <module>
    run(args)
  File "run.py", line 53, in run
    run_bsi_pt(args)
  File "/home/runner/work/bayesian-strategy-inference/bayesian-strategy-inference/src/baseball_game/run.py", line 648, in run_bsi_pt
    policy_preds.append(step_2_policy_preds[i])
IndexError: list index out of range

Instead of relying on where the user runs the script. Use git rev-parse --show-toplevel to get the full path of the top-level directory.

Add mypy in pre-commit.

Existing code hasn't go through code quality check yet, need to run per-commit hooks (including formatter and linter) once for all files.

https://jerry871002.github.io/bayesian-strategy-inference/environments/soccer_game/

In the last paragraph, the folder name is incorrect and should be changed from navigation_game to soccer_game.

https://jerry871002.github.io/bayesian-strategy-inference/environments/baseball_game/

In BaseballGame, the show() method requires a parameter (actions), which is weird because show() should print the state of the game at the moment and not a way to test the agent-opponent action combination.

Replace print with logging for better flexibility when managing logs.

As we are using this abbreviation everywhere (e.g., titles of the README and the documentation site), and full name bayesian-strategy-inference-policy-tracking is obviously too lengthy, better to change it to bsi-pt to keep things consistent.

Kueitso sent us the latest code he used to produce the results in the paper, and we should update the code here to match the newest version.

This issue requires careful inspection to make sure we keep everything working.

Tasks

• #1
• #17
• Integrate the baseball environment with run_exp_and_plot.py
• Use scripts/run_exps_and_plot.sh to run the baseball experiments

Some scripts in the scripts/ folder are no longer used. Remove them.

Add debug statements, such as the episode and step number, in baseball environment like in the existing environments.

Currently, we are using p_pattern to control the randomness of the new-phi-noise opponent.

p_pattern is the complement of the parameter $\epsilon$ (epsilon) described in the paper, i.e. p_pattern = 1 - epsilon. Consider removing p_pattern and using only epsilon to control the randomness to be consistent with the paper.

After #13 is merged, update the usage of scripts/run_exps_and_plot.sh according to the latest version.

Containerizing this project allows the user to run the experiments with the container runtime as the only dependency.

To achieve this, we have several tasks to do

• #8
• Create an image using Dockerfile. The image should be based on a Python runtime image and install all the required dependencies
• Run a container with the newly created image and make sure the experiment results are correct

In index.md (the corresponding webpage is https://jerry871002.github.io/bayesian-strategy-inference/), we suggest the user run scripts/run_exps_and_plot.sh [NUM_RUNS] [NUM_EPISODES] but didn't explain what the parameters mean.

run_experiment.py should work with the following agent-opponent combinations.

• BSI-PT vs PhiOpponent
• BSI-PT vs PhiNoiseOpponent
• OKR vs PhiOpponent
• OKR vs PhiNoiseOpponent
• Deep BPR+ vs PhiOpponent
• Deep BPR+ vs PhiNoiseOpponent
• BPR+ vs PhiOpponent
• BPR+ vs PhiNoiseOpponent