[BUG] rollout with non-uniform tensors no longer supported about rl HOT 7 CLOSED

Missed to include the versions:
0.4.0 1.26.2 3.10.13 (main, Sep 11 2023, 08:16:02) [Clang 14.0.6 ] darwin

from rl.

Yes unfortunately we deprecated the behaviour of stack because that required loads of checks and also some surprising behaviours from time to time!
You can also change this by passing the contiguous=False argument to rollout

Note to self: we should make it so that contiguous=False calls LazyStackedTensorDict.maybe_dense_stack to get the "best" stack possible

from rl.

Ok thanks that's helpful! Also, I wonder if we could create dense stacks of the nonuniform shaped tensors by batching them as nested tensors instead?

from rl.

Yes we can do that, I don't think we have the util to stack things as nested tensors yet though
The main issue we'll be facing is that nested tensors can only be stacked along dim 0 and rollout stacks along the last+1 dim, so it will only work for non-batched envs

from rl.

I wonder if we encounter such data maybe we can automatically pad the data and return a padding mask along the dynamic dimensions. This can be the fallback when the contained tensors are nested tensors. If you think that's an option, I can look into implementing something like this after the NeurIPS deadline.

from rl.

Another option is to create the nested tensor along dim 0 but "lie" with tensordict and present it as a transposed version
we'll need to patch the get set etc

from rl.

Closing this as now contiguous=False will do a best intention attempt to stack what can be stacked and return non-contiguous stacks everywhere else

from rl.