ggsding / sam-cd Goto Github PK

View Code? Open in Web Editor NEW

119.0 7.0 10.0 435 KB

Pytorch code of the SAM-CD

Python 100.00%

sam-cd's Introduction

SAM-CD

Pytorch codes of Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images [paper]

The SAM-CD adopts FastSAM as the visual encoder with some modifications.

2024-4-30 Update:

SAM-CD now supports access to efficientSAM. Check the updated model at models/effSAM_CD.py (prior installation of efficientSAM at the project folder is required). However, direct integration of efficientSAM may cause an accuracy drop, so there is space to further improve the SAM-CD architecture.

How to Use

Installation
- Install FastSAM following the instructions.
- Modify the Ultralytics source files following the instructions at: 'SAM-CD/models/FastSAM/README.md'.
Dataset preparation.
- Please split the data into training, validation and test sets and organize them as follows:

      YOUR_DATA_DIR
      ├── ...
      ├── train
      │   ├── A
      │   ├── B
      │   ├── label
      ├── val
      │   ├── A
      │   ├── B
      │   ├── label
      ├── test
      │   ├── A
      │   ├── B
      │   ├── label

Find change line 13 in SAM-CD/datasets/Levir_CD.py (or other data-loading .py files), change /YOUR_DATA_ROOT/ to your local dataset directory.

Training

classic CD training: python train_CD.py

training CD with the proposed task-agnostic semantic learning: python train_SAM_CD.py

line 16-45 are the major training args, which can be changed to load different datasets, models and adjust the training settings.
Inference and evaluation

inference on test sets: set the chkpt_path and run

python pred_CD.py

evaluation of accuracy: set the prediction dir and GT dir, and run

python eval_CD.py

(More details to be added...)

Dataset Download

In the following, we summarize links to some frequently used CD datasets:

Pretrained Models

For readers to easily evaluate the accuracy, we provide the trained weights of the SAM-CD.

Drive
Baidu (pswd: SMCD)

Cite SAM-CD

If you find this work useful or interesting, please consider citing the following BibTeX entry.

@article{ding2024adapting,
title={Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images},
author={Ding, Lei and Zhu, Kun and Peng, Daifeng and Tang, Hao and Yang, Kuiwu and Bruzzone, Lorenzo},
journal={IEEE Transactions on Geoscience and Remote Sensing}, 
year={2024},
volume={62},
pages={1-11},
doi={10.1109/TGRS.2024.3368168}
}

sam-cd's People

Contributors

Stargazers

Watchers

Forkers

sacrificeljj stephansturges westamine brustt wdc233 fengzhang2011 tmits37 chintyapraja yili9111 huili-1221

sam-cd's Issues

RuntimeError: shape '[1, 105, -1]' is invalid for input of size 10485760

Hello, your work is very interesting. Thank you for the open source as well as the guide to use it. Although I have followed the https://github.com/ggsDing/SAM-CD/blob/main/models/FastSAM/README.md , I am having some problems with it and I was wondering if it would be convenient for you to answer them. The questions are as follows:

File "/anaconda3/envs/FastSAM/lib/python3.9/site-packages/ultralytics/nn/tasks.py", line 96, in _predict_once
x = m(x) # run
File "/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/anaconda3/envs/FastSAM/lib/python3.9/site-packages/ultralytics/nn/modules/head.py", line 94, in forward
x = self.detect(self, x)
File "/anaconda3/envs/FastSAM/lib/python3.9/site-packages/ultralytics/nn/modules/head.py", line 54, in forward
x_cat = torch.cat([xi.view(shape[0], self.no, -1) for xi in x], 2)
File "/anaconda3/envs/FastSAM/lib/python3.9/site-packages/ultralytics/nn/modules/head.py", line 54, in
x_cat = torch.cat([xi.view(shape[0], self.no, -1) for xi in x], 2)
RuntimeError: shape '[1, 105, -1]' is invalid for input of size 10485760

Thanks for reply,but the WHU-CD dataset, I can not open. Just like this

如何调整FastSAM第一层输出特征L1的大小

作者你好，你所提出的SAM-CD对遥感变化检测领域非常有意义。
我有一些问题想请教一下，论文中SAM-CD第一个特征层输出的特征L1为输入图像的1/4，我想将第一个特征层输出的特征L1改为输入图像的1/2。针对上述问题，在源码中，我应该如何更改。

code about pred_CD & eval_CD when classic CD training

Hi, firstly a nice work!
I wanna to ask when I just use classic CD training: python train_CD.py , how to change pred_CD.py and eval_CD.py?
Can you provide another version?
Thanks a lot!

results on the LEVIR-CD

Thank you very much for your excellent work，

I found that I could not achieve the expected results of the paper when reproducing the code results. May I ask if there are any issues with my reproduction process？

args = {
    'train_batch_size': 4,
    'val_batch_size': 4,
    'lr': 0.1,
    'epochs': 50,
    'gpu': True,
    'dev_id': 0,
    'multi_gpu': None,  #"0,1,2,3",
    'weight_decay': 5e-4,
    'momentum': 0.9,
    'print_freq': 50,
    'predict_step': 5,
    'crop_size': 512,
    'pred_dir': os.path.join(working_path, 'results', DATA_NAME),
    'chkpt_dir': os.path.join(working_path, 'checkpoints', DATA_NAME),
    'log_dir': os.path.join(working_path, 'logs', DATA_NAME, NET_NAME),
    'load_path': os.path.join(working_path, 'checkpoints', DATA_NAME, 'xxx.pth')}

python train_SAM_CD.py

reproduce results:
paper results

'Result' object has no attribute 'clone'.

在训练过程中报以下错误：
File "/SAM-CD/models/SAM_CD.py", line 165, in forward
featA_s4 = self.Adapter4(featsA[3].clone())
File "//anaconda3/envs/pytorch17/lib/python3.8/site-packages/ultralytics/yolo/utils/init.py", line 135, in getattr
raise AttributeError(f"'{name}' object has no attribute '{attr}'. See valid attributes below.\n{self.doc}")
AttributeError: 'Results' object has no attribute 'clone'.
想问下作者在实现过程中是否遇见。想想！

train code with SAM

Hi there, sorry to bother you again!
Recently I'm replicating you model. I want to ask how to perform SAM with your model instead of FastSAM. Can you provide an instruction?
My mail is [email protected].
Thanks a lot!

Inference problems

I am trying to run the inference, but I encounter a problem. This line list(self.stream_inference(source, model)) in \ultralytics\yolo\engine\predictor.py returns an empty list, although the elements of the generator object are non empty. Could you please provide a solution?

Have you tried training with mobile-sam?

Have you tried using mobile-sam in Ultralytics? Does it support access to low-level features? I've noticed that the training process is quite lengthy with a large dataset. Currently, I'm unsure if mobile-sam can improve training efficiency compared to fastsam.

cpu版本

很感谢您提供优秀的开源代码，请问可以在readme中介绍如何使用cpu版本吗，非常感谢。

Good paper, when are you open source the code?

Dataset issue, hello , I wanna know where I can get the WHU-CD dataset. Could you please offer a Link?

Since I can find the LEVIR-CD Dataset, I wanna use the other dataset . But I can not find it .

AttributeError: 'tuple' object has no attribute 'sigmoid'

Do I need change the output tuple to the outC only?

Thanks a lot.

关于数据集的问题。

请问使用不同数据集需要注意什么呢？
我能够基于Levir-cd使用https://github.com/ggsDing/SAM-CD/blob/main/train_SAM_CD.py，但是当我更换别的数据集的时候，会出现以下错误：

Traceback (most recent call last):
File "/home/server/SAM-CD/train_SAM_CD.py", line 202, in
main()
File "/home/server/SAM-CD/train_SAM_CD.py", line 68, in main
train(train_loader, net, optimizer, val_loader)
File "/home/server/SAM-CD/train_SAM_CD.py", line 91, in train
for i, data in enumerate(train_loader):
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 628, in next
data = self._next_data()
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1333, in _next_data
return self._process_data(data)
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1359, in _process_data
data.reraise()
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/_utils.py", line 543, in reraise
raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 1.
Original Traceback (most recent call last):
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 61, in fetch
return self.collate_fn(data)
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
return collate(batch, collate_fn_map=default_collate_fn_map)
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 143, in collate
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 143, in
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 120, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 172, in collate_numpy_array_fn
return collate([torch.as_tensor(b) for b in batch], collate_fn_map=collate_fn_map)
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 120, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/home/server/anaconda3/envs/FastSAM/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 163, in collate_tensor_fn
return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [512, 512] at entry 0 and [512, 512, 3] at entry 2

多卡训练的问题

想试图用多卡训练提速，修改参数后会报以下错误
……
……
900/1024 images loaded.
950/1024 images loaded.
1000/1024 images loaded.
(256, 256, 3)
1024 val images loaded.
Segmentation fault (core dumped)

修改了 run_encoder模块后还是报同样错误

预训练模型

您好，请问有预训练权重嘛？

Samcd

这里显示的是results形式的输出，然后在下一步就报错。大佬这是咋回事啊

A Error.

I can run the cd. But when i run the sam-cd model, there are some error in model. Hope your answer. I debug the code, the len(featsA) is 2.

Traceback (most recent call last):
File "E:/code/SAM-CD-main/train_SAM_CD.py", line 202, in
main()
File "E:/code/SAM-CD-main/train_SAM_CD.py", line 68, in main
train(train_loader, net, optimizer, val_loader)
File "E:/code/SAM-CD-main/train_SAM_CD.py", line 101, in train
outputs, outA, outB = net(imgs_A, imgs_B)
File "D:\Program Files (x86)\anaconda3\envs\SAM\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "E:\code\SAM-CD-main\models\SAM_CD.py", line 159, in forward
featA_s4 = self.Adapter4(featsA[3].clone())
IndexError: list index out of range