[BUG] Ray's actor construction is broken in docstring about rl HOT 8 OPEN

Cc @albertbou92

from rl.

I am able to reproduce the bug. Let me look into it.

from rl.

I have been further able to reproduce the bug with some code that does not involve TorchRL. TorchRL collectors inherit from IterableDataset, which apparently with this python version raises the bug when calling ray.remote.

import ray
from torch.utils.data import IterableDataset


class SyncDataCollector(IterableDataset):
    def __iter__(self):
        return


ray.init()
ray.remote(SyncDataCollector)

Digging a bit more seems related to the way ray handles classes to turn them to remote classes
https://github.com/ray-project/ray/blob/master/python/ray/actor.py#L1606C13-L1606C26
I have opened an issue for now

from rl.

ray-project/ray#42914

from rl.

I tried with python 3.8 and I don't get the GenericAlias issue, but I got a different error:

2024-02-12 20:29:01,299 INFO worker.py:1724 -- Started a local Ray instance.
Traceback (most recent call last):
  File "sample_envs.py", line 55, in <module>
    distributed_collector = RayCollector(
  File "3.8_torchrl\lib\site-packages\torchrl\collectors\distributed\ray.py", line 475, in __init__
    ray.wait(object_refs=pending_samples)
  File "3.8_torchrl\lib\site-packages\ray\_private\auto_init_hook.py", line 22, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "3.8_torchrl\lib\site-packages\ray\_private\client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
TypeError: wait() got an unexpected keyword argument 'object_refs'

from rl.

I was able to reproduce the error. Ray changed the signature of ray.wait() in recent versions. This PR (https://github.com/pytorch/rl/pull/1908/files) fixes the problem. The PR makes the ray example code run without errors. Thanks a lot for bringing the issue to our attention.

I will also monitor the open issue in the official Ray repository for any updates aimed at resolving compatibility issues with newer Python versions.

from rl.

@albertbou92 Should we close this or is there anything outstanding?

from rl.

I would keep the issue open until ray-project/ray#42914 is solved. atm only works for Python 3.8 or lower.

from rl.