Giter Club home page Giter Club logo

seglord's Introduction

SegLord: Segmentation Lord

Installation

Environment Setup

python3 -m venv .env
source .env/bin/activate
python -m pip install -U pip

Package Installation

bash install.sh

OR

mkdir .cache

TMPDIR=./.cache pip install wheel tqdm wandb
TMPDIR=./.cache pip3 install torch torchvision torchaudio
TMPDIR=./.cache pip install accelerate einops
TMPDIR=./.cache pip install albumentations
TMPDIR=./.cache pip install segmentation_models_pytorch
TMPDIR=./.cache pip install torchmetrics
TMPDIR=./.cache pip install segformer-pytorch

Experiment Conduction

This repo training procedure is built with the support of Accelerator, thus enabling various modes of training. Before training, direct the working folder to path/to/seglord/seglord. There are two main ways for running with Accelerator

accelerate launch {script_name.py} --arg1 --arg2 ...

or 

python -m accelerate.commands.launch --num_processes=2 {script_name.py} {--arg1} {--arg2}

Single GPU

CUDA_VISIBLE_DEVICES={GPU_ID} accelerate launch main.py --ds citynormal --model dl3p --loss dice --wandb 

or

accelerate launch --gpu_ids {GPU_ID} main.py --ds citynormal --model dl3p --loss dice --wandb 

Multi GPUs

To use all available GPUs

accelerate launch --multi_gpu {GPU_ID} main.py --ds citynormal --model dl3p --loss dice --wandb 

Or to specify the number of GPUs in training

accelerate launch --num_processes=2 main.py --ds citynormal --model dl3p --loss dice --wandb

CPU

To use CPU for training

accelerate launch --cpu main.py --ds citynormal --model dl3p --loss dice --wandb

Precision Configuration

accelerate launch --multi_gpu --mixed_precision=fp16 --num_processes=2 main.py --ds citynormal --model dl3p --loss dice --wandb

Run from configs

There are templates for config file at path/to/seglord/seglord/scripts.

Single GPU

CUDA_VISIBLE_DEVICES="0" accelerate launch --config_file ./scripts/single_gpu.yaml main.py --epochs 1 --debug --wandb

Multi GPU

accelerate launch --config_file ./scripts/multi_gpu.yaml main.py --epochs 1 --debug --wandb

seglord's People

Contributors

khoidoo avatar

Watchers

 avatar

Forkers

skydvn

seglord's Issues

AttributeError: 'GeneralTracker' object has no attribute 'run'

Total number of Params: 14329972
Total number of Params: 14329972
Device in use: cuda:0
Device in use: cuda:1
wandb: Currently logged in as: khoido8899 (truelove). Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.17.0
wandb: Run data is saved locally in /media/mountHDD2/git/seglord/seglord/wandb/run-20240527_214257-kmam313l
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run 05_27_2024_21_42_56
wandb: โญ๏ธ View project at https://wandb.ai/truelove/seglord
wandb: ๐Ÿš€ View run at https://wandb.ai/truelove/seglord/runs/kmam313l
Epoch: 0 - train/loss: 0.93 - train/miou: 0.03 - train/dice: 0.00 - valid/loss: 1.02 - valid/miou: 0.02 - valid/dice: 0.00: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 1/1 [01:30<00:00, 90.95s/it]
[rank1]: Traceback (most recent call last):
[rank1]:   File "/media/mountHDD2/git/seglord/seglord/main.py", line 147, in <module>
[rank1]:     main()
[rank1]:   File "/media/mountHDD2/git/seglord/seglord/main.py", line 142, in main
[rank1]:     evaluator.sync(accelerator=accelerator)
[rank1]:   File "/media/mountHDD2/git/seglord/seglord/metrics/eval.py", line 208, in sync
[rank1]:     wandb_tracker.run.log_artifact(best_model_art)
[rank1]: AttributeError: 'GeneralTracker' object has no attribute 'run'
W0527 21:44:32.935000 140080319242752 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1377426 closing signal SIGTERM
E0527 21:44:33.149000 140080319242752 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: 1) local_rank: 1 (pid: 1377427) of binary: /media/mountHDD2/git/seglord/.env/bin/python
Traceback (most recent call last):
  File "/media/mountHDD2/git/seglord/.env/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/media/mountHDD2/git/seglord/.env/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 46, in main
    args.func(args)
  File "/media/mountHDD2/git/seglord/.env/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1073, in launch_command
    multi_gpu_launcher(args)
  File "/media/mountHDD2/git/seglord/.env/lib/python3.10/site-packages/accelerate/commands/launch.py", line 718, in multi_gpu_launcher
    distrib_run.run(args)
  File "/media/mountHDD2/git/seglord/.env/lib/python3.10/site-packages/torch/distributed/run.py", line 870, in run
    elastic_launch(
  File "/media/mountHDD2/git/seglord/.env/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/media/mountHDD2/git/seglord/.env/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 263, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
main.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-05-27_21:44:32
  host      : exx-desktop
  rank      : 1 (local_rank: 1)
  exitcode  : 1 (pid: 1377427)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.