Comments (3)
If I understand correctly, does this mean that the "parameters" can be updated dynamically after the envs are initialized? Is it required to vary or randomize the "parameters" for the different envs in the pool?
One mechanism we have used internally at Sea AI lab is to pass a python function to the config, so that each time the envs reset, they dynamically generates a new config value as per specified in the python function. Would this mechanism suffice the use case here?
from envpool.
Thanks for the reply! Yes, we should not only be able to update the parameters for each episode, but even in each env step! Basically, I believe the mechanism you mention can handle this case (if it is efficient enough). But in terms of system design, I guess we can refer to the dm_control base env class.
You can see the functions like before_step
and after_step
, which will handle some parameter updating in a more pretty way. Besides, if we consider using a memory pool as the observation and action buffer, or define some complex reward computations, they should be updated in after_step
. Robotics guys split the step
only because there might be so many logic modules in each step. And these kinds of hooks are used in other popular robotics simulators like Raisim and IsaacGym. So together with #59, we may consider adding some new general modules for RL+Robotics usage.
from envpool.
Thanks for clarifying. It seems more profound that I initially thought. From your description, there're some complications mainly due to performance.
Passing a custom python function down to c++ is okay if we're doing it once in a while, e.g. using it to config an episode. But not ok for performance reason if it is done every step. C++ calling python is expensive and not parallelizable due to GIL. This will diminish the acceleration brought by envpool, and make it no faster than the other python based envs.
There're some work-arounds in my mind,
- Instead of letting the user fully customize it, we may implement a few c++ primitives. The user can pass parameters to select which primitive is called.
- Think out loud, we may limit the primitives to be implemented in numba, and called the jitted code from the c++ threads. (feasibility & efficiency unknown).
from envpool.
Related Issues (20)
- [BUG] No Envs after installing from sources HOT 19
- metaclass conflict HOT 1
- [BUG] Unable to build from source HOT 6
- [Feature Request] Add Atari difficulty level and a game mode option. HOT 3
- [BUG] Build failes due to Bazel 7.0.0 breaking change
- output 'envpool/mujoco/assets_gym' of //envpool/mujoco:gen_mujoco_gym_xml is a directory; dependency checking of directories is unsound
- [Feature Request] A simple (and effective?) way to support cherry-picked env reset in `xla` mode HOT 1
- [Feature Request] Get RAM State from Atari ALE HOT 5
- [BUG] Error running mujoco-gym tasks with the parameter xml_file HOT 1
- [BUG] incorrect parsing of actions in multiagent environment
- [Feature Request]Set random seed to each env HOT 1
- [BUG] Episode return is not recorded correctly in cleanRL's example
- Opinions on running envpool on a dedicated simulator server with e.g. REST API HOT 3
- [BUG] Atari Breakout does not reset with episodic_live=True
- What's the mean of `timestep.observation.players`?
- [BUG] Using XLA crashes out
- [Feature Request] Have a render() function exposed for Gym environments to see agent performance
- [Feature Request] Support for DM Lab2d Environments
- [Feature Request] Do this project support like gym.vector.asyncvecenv? HOT 3
- [BUG] Minigrid is not supported as per documentation
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from envpool.