RuntimeError: DataLoader worker exited unexpectedly about openpcdet HOT 4 CLOSED

It seems like a environment problem, could you run the test codes with given pretrained models successfully?

from openpcdet.

Hello, I am facing the same problem as well,
when trying to run the train.py or test.py on a headless server.
I get the same error :
2022-05-13 15:19:20,494 INFO Loading KITTI dataset
2022-05-13 15:19:20,597 INFO Total samples for KITTI dataset: 3769
/nfs/home/sc98255/.local/share/virtualenvs/OpenPCDet-9UU-mtWz/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
2022-05-13 15:19:24,554 INFO ==> Loading parameters from checkpoint pv_rcnn_8369.pth to CPU
2022-05-13 15:19:24,631 INFO ==> Done (loaded 367/367)
2022-05-13 15:19:24,649 INFO *************** EPOCH 8369 EVALUATION *****************
eval: 0%| | 0/3769 [00:00<?, ?it/s]None
None
None
None
Traceback (most recent call last):
File "/nfs/home/sc98255/.local/share/virtualenvs/OpenPCDet-9UU-mtWz/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 990, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "/usr/lib/python3.8/queue.py", line 179, in get
self.not_empty.wait(remaining)
File "/usr/lib/python3.8/threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
File "/nfs/home/sc98255/.local/share/virtualenvs/OpenPCDet-9UU-mtWz/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
_error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 1039472) exited unexpectedly with exit code 1. Details are lost due to multiprocessing. Rerunning with num_workers=0 may give better error trace.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "test.py", line 200, in
main()
File "test.py", line 196, in main
eval_single_ckpt(model, test_loader, args, eval_output_dir, logger, epoch_id, dist_test=dist_test)
File "test.py", line 62, in eval_single_ckpt
eval_utils.eval_one_epoch(
File "/nfs/home/sc98255/OpenPCDet/tools/eval_utils/eval_utils.py", line 54, in eval_one_epoch
for i, batch_dict in enumerate(dataloader):
File "/nfs/home/sc98255/.local/share/virtualenvs/OpenPCDet-9UU-mtWz/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/nfs/home/sc98255/.local/share/virtualenvs/OpenPCDet-9UU-mtWz/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1186, in _next_data
idx, data = self._get_data()
File "/nfs/home/sc98255/.local/share/virtualenvs/OpenPCDet-9UU-mtWz/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1142, in _get_data
success, data = self._try_get_data()
File "/nfs/home/sc98255/.local/share/virtualenvs/OpenPCDet-9UU-mtWz/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1003, in _try_get_data
raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
RuntimeError: DataLoader worker (pid(s) 1039472, 1039504) exited unexpectedly
eval: 0%|

When I put workers 0 it stops at eval 0%

Did you guys solve it ?

from openpcdet.

This is what I obtain when using 0 workers
If you could help me, I am stuck, same thing goes for testing

from openpcdet.

Hey, I am facing the same error. I am not able to identify the issue. Were you people able to solve it @udacityyy and @SlingHe ? If yes, how?

Thanks.

from openpcdet.