Comments (5)
Trinkle23897
commented on August 15, 2024
1
Yep, will do.
from envpool.
Trinkle23897
commented on August 15, 2024
I don't think this is a bug in envpool. Actually, this is a new feature in gym.vector_env, but there is another way to handle this issue. Let me post an example of Catch-v0 in #34
from envpool.
Trinkle23897
commented on August 15, 2024
In [18]: import envpool
In [19]: import numpy as np
In [20]: e = envpool.make_gym("Catch-v0")
In [21]: e.reset()
Out[21]:
array([[[0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32)
In [22]: e.step(np.array([1]))
Out[22]:
(array([[[0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [23]: e.step(np.array([1]))
Out[23]:
(array([[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [24]: e.step(np.array([1]))
Out[24]:
(array([[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [25]: e.step(np.array([1]))
Out[25]:
(array([[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [26]: e.step(np.array([1]))
Out[26]:
(array([[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [27]: e.step(np.array([1]))
Out[27]:
(array([[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [28]: e.step(np.array([1]))
Out[28]:
(array([[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [29]: e.step(np.array([1]))
Out[29]:
(array([[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [30]: e.step(np.array([1]))
Out[30]:
(array([[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 1., 1., 0., 0.]]], dtype=float32),
array([-1.], dtype=float32),
array([ True]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})
In [31]: e.step(np.array([1]))
Out[31]:
(array([[[0., 0., 0., 1., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 1., 0., 0.]]], dtype=float32),
array([0.], dtype=float32),
array([False]),
{'env_id': array([0], dtype=int32),
'players': {'env_id': array([0], dtype=int32)},
'TimeLimit.truncated': array([False])})from envpool.
Trinkle23897
commented on August 15, 2024
In 30 it reaches the terminal state, but it doesn't trigger the auto-reset. Instead, in 31 it sends another action and triggers the auto-reset. At this step the action will be ignored by env.
from envpool.
araffin
commented on August 15, 2024
i see, i think that would be important to document it as it can result in off by one error, see takuseno/d3rlpy#98 (comment)
from envpool.
Related Issues (20)
- [BUG] wrong handle in example? HOT 2
- [Question] Does EnvPool support Procgen's Exploration mode
- [Feature Request] 请问什么时候才能在windows上使用呀
- Does envpool support arm architecture？ HOT 2
- [BUG] AssertionError: MiniGrid-Empty-5x5-v0 is not supported HOT 2
- Set values for action in C++ HOT 2
- [BUG] XLA interface does not work with jax>=0.4.16
- [BUG] XLA Segmentation Fault HOT 8
- [BUG] No Envs after installing from sources HOT 19
- metaclass conflict HOT 1
- [BUG] Unable to build from source HOT 6
- [Feature Request] Add Atari difficulty level and a game mode option. HOT 3
- [BUG] Build failes due to Bazel 7.0.0 breaking change
- output 'envpool/mujoco/assets_gym' of //envpool/mujoco:gen_mujoco_gym_xml is a directory; dependency checking of directories is unsound
- [Feature Request] A simple (and effective?) way to support cherry-picked env reset in `xla` mode HOT 1
- [Feature Request] Get RAM State from Atari ALE HOT 5
- [BUG] Error running mujoco-gym tasks with the parameter xml_file HOT 1
- [BUG] incorrect parsing of actions in multiagent environment
- [Feature Request]Set random seed to each env HOT 1
- [BUG] Episode return is not recorded correctly in cleanRL's example
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from envpool.