awslabs / amazon-accessible-rl-sdk Goto Github PK

Applying my own data to the BYO Planner code it fails for the Behavioural Clone Planner part, specifically in this line of code:

immediate_cost = df_sar[tokenizer.df_tokenized.rewards[:-1]].values

The reason why it fails is because it assumes that tokenizer.df_tokenized.rewards is a list with more than one element and it can therefore apply [:-1]. In my case though:

len(tokenizer.df_tokenized.rewards)
1

which leads to:

tokenizer.df_tokenized.rewards[:-1]
[]

and finally:

df_sar[tokenizer.df_tokenized.rewards[:-1]].values
array([], shape=(1, 0), dtype=float64)

Should this be instead something like:

immediate_cost = immediate_cost = df_sar[tokenizer.df_tokenized.rewards[:-1 if len(tokenizer.df_tokenized.rewards)>1 else 0]].values

Propose to remove stable_baseline3 from requirements.txt, and move the affected implementations out from a2rl to the examples.

stable_baseline3 depends on gym-0.21 which conflicts to the newer version that a2rl requires. The old gym version doesn't come with complete type checks, and still with deprecated or removed gym APIs.

Currently, a2rl requires stable_baseline3 at these places:

• in data_generator_gym() which is used only in data_properties.ipynb.
• in the docstring of WhatifWrapper.

In all cases above, the examples still work (fortunately) with a newer gym.

[UPDATE] Alternatively, investigate whether it's possible to switch to mushroom-rl. It supports newer gym, but due to Cython requirement, need to check how easy is installation on ARM and DL1 instances.