Classic environments for reinforcement learning and dynamic programming, implemented in OpenAI Gym and Gymnasium.
Home Page: https://pypi.org/project/gym-classics/
License: GNU General Public License v3.0
To match Example 6.6 on page 132 in Sutton & Barto (2018), entering the cliff should just teleport the agent to the start without actually terminating the episode. Termination should occur only when the agent reaches the goal.
Currently, both Racetrack environments terminate when the agent lands on the finish line (see code), but the textbook defines termination to be when the agent crosses the finish line.
I need to double check the implementation, but I think this implies that any action that lands the agent to the right of a finish-line cell should be considered successful. I noticed this discrepancy when my Monte Carlo method was having difficulty reaching the finish line with random exploration.
TODO:
_done method in abstract Racetrack classRight now, when I call the example code:
import gym # version 0.26.2
import gym_classics
env = gym.make('ClassicGridworld-v0')
state = env.reset()
/usr/local/lib/python3.10/site-packages/gym/utils/passive_env_checker.py:174: UserWarning: WARN: Future gym versions will require that `Env.reset` can be passed a `seed` instead of using `Env.seed` for resetting the environment random number generator.
logger.warn(
/usr/local/lib/python3.10/site-packages/gym/utils/passive_env_checker.py:187: UserWarning: WARN: Future gym versions will require that `Env.reset` can be passed `options` to allow the environment initialisation to be passed additional information.
logger.warn(
/usr/local/lib/python3.10/site-packages/gym/utils/passive_env_checker.py:195: UserWarning: WARN: The result returned by `env.reset()` was not a tuple of the form `(obs, info)`, where `obs` is a observation and `info` is a dictionary containing additional information. Actual type: `<class 'int'>`
logger.warn(
>>>
I will get some warnings. Possibly this can be resolved by moving from gym to gymnasium? However, right now just replacing gym by gymnasium does not work:
import gymnasium as gym
import gym_classics
env = gym.make('ClassicGridworld-v0')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.10/site-packages/gymnasium/envs/registration.py", line 592, in make
_check_version_exists(ns, name, version)
File "/usr/local/lib/python3.10/site-packages/gymnasium/envs/registration.py", line 218, in _check_version_exists
_check_name_exists(ns, name)
File "/usr/local/lib/python3.10/site-packages/gymnasium/envs/registration.py", line 195, in _check_name_exists
raise error.NameNotFound(
gymnasium.error.NameNotFound: Environment ClassicGridworld doesn't exist.
My guess is that calling import gym_classics will internally call import gym and register the environment in the older version, so the environments wont appear in gymnasium?
Possible solution: can you check whether gym has been already loaded, before loading it yourself in gym_classics. Can that do the trick already?
First of all: THANK YOU for the really nice toolbox!
I have some issue with the following code:
import gym # version 0.26.2
import gym_classics
env = gym.make('ClassicGridworld-v0')
state = env.reset()
env.step(1)
Running python (3.9, 3.10, 3.11) I get some error like:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 37, in step
return self.env.step(action)
File "/usr/local/lib/python3.9/site-packages/gym/wrappers/env_checker.py", line 37, in step
return env_step_passive_checker(self.env, action)
File "/usr/local/lib/python3.9/site-packages/gym/utils/passive_env_checker.py", line 214, in env_step_passive_checker
result = env.step(action)
File "/usr/local/lib/python3.9/site-packages/gym_classics/envs/abstract/base_env.py", line 67, in step
elements = self._sample_random_elements(state, action)
File "/usr/local/lib/python3.9/site-packages/gym_classics/envs/abstract/noisy_gridworld.py", line 14, in _sample_random_elements
return [self._noisy_action(action)]
File "/usr/local/lib/python3.9/site-packages/gym_classics/envs/abstract/noisy_gridworld.py", line 23, in _noisy_action
p = self.np_random.rand()
AttributeError: 'numpy.random._generator.Generator' object has no attribute 'rand'
I guess it can be easily solved by changing:
File "/usr/local/lib/python3.9/site-packages/gym_classics/envs/abstract/noisy_gridworld.py", line 23, in _noisy_action
p = self.np_random.rand()
to
p = self.np_random.random()
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.