The lightning-template from tshu-w

Hi there 👋

🧑🏻‍🎓 PhD candidate at UCAS
🔭 Currently interested in LLM for Data Science (Entity Resolution and Data Analytics)
🛠 macOS & Emacs & ZSH & Rime
🔑 GPG Public Key

GitHub Stats

lightning-template's People

Contributors

Stargazers

Watchers

lightning-template's Issues

__resolve_ckpt_dir does not work with ddp

Hi @tshu-w ,

at first, thank you very much for providing and maintaining this repo!

I just noticed, that the __resolve_ckpt_dir function brakes when used in conjunction with ddp.

I am on c65989f and created the environment with:

mamba env create -n lt -f environment.yaml and just tried:

./run fit -c ./configs/mnist.yaml -c ./configs/presets/ddp.yaml -c ./configs/presets/debugger.yaml

Error message

...
  File "/home/jwkuenzel/miniconda3/envs/lt/lib/python3.10/site-packages/lightning/pytorch/trainer/call.py", line 190, in _call_callback_hooks
    fn(trainer, trainer.lightning_module, *args, **kwargs)
  File "/home/jwkuenzel/miniconda3/envs/lt/lib/python3.10/site-packages/lightning/pytorch/callbacks/model_checkpoint.py", line 257, in setup
    dirpath = self.__resolve_ckpt_dir(trainer)
  File "/home/jwkuenzel/projects/lightning-template/src/utils/loggers.py", line 33, in __resolve_ckpt_dir
    ckpt_path = os.path.join(trainer.log_dir, "checkpoints")
  File "/home/jwkuenzel/miniconda3/envs/lt/lib/python3.10/posixpath.py", line 76, in join
    a = os.fspath(a)
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
TypeError: expected str, bytes or os.PathLike object, not NoneType
    a = os.fspath(a)
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
TypeError: expected str, bytes or os.PathLike object, not NoneType

Recommend Projects