Comments (1)
No plan to do this in the near future after all, for the following reason:
When querying a value network, the output will be a *shape, 1
tensor. To keep the ProbabilisticOperator
class general, it feels that setting some option to squeeze the last dimension would be overkilling it. Therefore, values will have a singleton dimension.
Because of this, if reward and done don't have this singleton dimension, we will end up having to squeeze the value to match the reward, or unsqueeze the reward to match the value. Hence, we will end up with as many squeeze/unsqueeze operations as before.
Having the rule that every tensor in a tensordict has a at least one more dimension than the tensordict does not seem too restrictive for now.
from rl.
Related Issues (20)
- [Question] Item called in loss modules forces synchronization? HOT 1
- [Feature Request] TorchRL to gym API HOT 1
- [BUG] ImportError: cannot import name 'MemoryMappedTensor' from 'tensordict' HOT 2
- [BUG] An attempt has been made to start a new process before the current process has finished its bootstrapping phase.
- [BUG] `TransformedEnv` does not copy env properties HOT 2
- [Feature Request] Pass additional information into the tensordict returned by the rollout method (EnvBase) HOT 3
- [Feature Request] Refactoring of TensorSpec and documentation HOT 2
- [BUG] The collector expands its container tensordict on the rollout device before moving it to the storing device. HOT 3
- [BUG] RuntimeError An attempt has been made to start a new process before the current process has finished its bootstrapping phase HOT 3
- [BUG] VD4RL Torchrl data omission HOT 2
- [BUG] StepCounter overrides native truncation. HOT 3
- [BUG] Shape Mismatch with default_info_dict_reader HOT 2
- [Feature Request] Simplify the example and tutorial codes HOT 4
- [Feature Request] Muzero and MCTS implementations HOT 1
- [BUG] Parallel envs override each others' num threads HOT 1
- [BUG] `check_env_specs` shouldn't set `torch.manual_seed` by default. HOT 3
- [BUG] Ray's actor construction is broken in docstring HOT 8
- [BUG] Problems with BatchedEnv on accelerated device with single envs on cpu HOT 28
- [BUG] Truncated key is on different devices with BatchedEnv on different device than single envs. HOT 2
- [Feature Request] partial steps in batches envs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rl.