justchenhao / stanet Goto Github PK
View Code? Open in Web Editor NEWofficial implementation of the spatial-temporal attention neural network (STANet) for remote sensing image change detection
License: BSD 2-Clause "Simplified" License
official implementation of the spatial-temporal attention neural network (STANet) for remote sensing image change detection
License: BSD 2-Clause "Simplified" License
BASE:
(epoch: 196) Overall_Acc: 0.973 Mean_IoU: 0.787 0: 0.972 1: 0.602 precision_1: 0.612 recall_1: 0.972 F1_1: 0.751
================ val acc (Tue Jul 21 23:39:54 2020) ================
(epoch: 197) Overall_Acc: 0.972 Mean_IoU: 0.784 0: 0.971 1: 0.596 precision_1: 0.607 recall_1: 0.971 F1_1: 0.747
================ val acc (Tue Jul 21 23:40:53 2020) ================
(epoch: 198) Overall_Acc: 0.973 Mean_IoU: 0.786 0: 0.972 1: 0.600 precision_1: 0.612 recall_1: 0.970 F1_1: 0.750
================ val acc (Tue Jul 21 23:41:53 2020) ================
(epoch: 199) Overall_Acc: 0.968 Mean_IoU: 0.766 0: 0.967 1: 0.565 precision_1: 0.572 recall_1: 0.977 F1_1: 0.722
================ val acc (Tue Jul 21 23:42:53 2020) ================
(epoch: 200) Overall_Acc: 0.970 Mean_IoU: 0.773 0: 0.969 1: 0.577 precision_1: 0.585 recall_1: 0.974 F1_1: 0.731
PAM:
================ val acc (Sun Jul 19 21:39:19 2020) ================
(epoch: 196) Overall_Acc: 0.976 Mean_IoU: 0.532 0: 0.976 1: 0.087 precision_1: 0.466 recall_1: 0.097 F1_1: 0.161
================ val acc (Sun Jul 19 21:39:40 2020) ================
(epoch: 197) Overall_Acc: 0.976 Mean_IoU: 0.534 0: 0.976 1: 0.092 precision_1: 0.443 recall_1: 0.104 F1_1: 0.168
================ val acc (Sun Jul 19 21:40:01 2020) ================
(epoch: 198) Overall_Acc: 0.976 Mean_IoU: 0.528 0: 0.976 1: 0.079 precision_1: 0.502 recall_1: 0.086 F1_1: 0.147
================ val acc (Sun Jul 19 21:40:23 2020) ================
(epoch: 199) Overall_Acc: 0.976 Mean_IoU: 0.535 0: 0.975 1: 0.095 precision_1: 0.424 recall_1: 0.109 F1_1: 0.174
================ val acc (Sun Jul 19 21:40:45 2020) ================
(epoch: 200) Overall_Acc: 0.976 Mean_IoU: 0.529 0: 0.976 1: 0.081 precision_1: 0.476 recall_1: 0.089 F1_1: 0.150
from models.base_model import BaseModel
File "./STANet-master/models/base_model.py", line 4, in
from abc import ABC, abstractmethod
ImportError: cannot import name ABC
how to solve it?
Hello!
I would like to know if the images feed to the neural network need to be in 1024x1024, like LEVIR-CD, or I have to create a folder and crop the A, B and label images in 256x256? Or your code already does this?
Thanks in advance!
I train your code,but get precision_1 is about 6.3 whatever base,bam,pam.
base:
================ val acc (Sun Jun 14 01:54:07 2020) ================
(epoch: 200) Overall_Acc: 0.975 Mean_IoU: 0.796 0: 0.974 1: 0.617 precision_1: 0.633 recall_1: 0.962 F1_1: 0.763
bam:
================ val acc (Sat Jun 13 18:21:13 2020) ================
(epoch: 200) Overall_Acc: 0.975 Mean_IoU: 0.794 0: 0.974 1: 0.615 precision_1: 0.630 recall_1: 0.963 F1_1: 0.762
pam:
================ val acc (Sun Jun 14 06:59:22 2020) ================
(epoch: 200) Overall_Acc: 0.975 Mean_IoU: 0.794 0: 0.974 1: 0.614 precision_1: 0.629 recall_1: 0.963 F1_1: 0.761
HI congratulations
thanks a lot for your code
I learn lot of things about your code
感谢您的工作。
请问 BAM 和 PAM 的模型参数量和 FLOPs 情况如何呢?
Hello, I would like to know how it would be possible to test without the label layer, that is, only the image from before and the image from after.
I imagine the code could be modified to do this.
Thank you very much for your attention . I await your response .
@justchenhao Are you planning to release pretrained weights?
Traceback (most recent call last): File "train.py", line 142, in <module> model.optimize_parameters() # calculate loss functions, get gradients, update network weights File "/home/user/zxm/stanet/STANet-master1/STANet-master/models/CDFA_model.py", line 122, in optimize_parameters self.backward() # calculate graidents for G File "/home/user/zxm/stanet/STANet-master1/STANet-master/models/CDFA_model.py", line 108, in backward self.loss.backward() File "/home/user/anaconda3/lib/python3.6/site-packages/torch/tensor.py", line 118, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/home/user/anaconda3/lib/python3.6/site-packages/torch/autograd/__init__.py", line 93, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: CUDA out of memory. Tried to allocate 2.00 GiB (GPU 3; 7.43 GiB total capacity; 5.01 GiB already allocated; 1.92 GiB free; 12.36 MiB cached)
First of all congratulate you on the great work done, the code is wonderful.
My question is about what differences exist between the three different training methods( PAM,BAM , etc) and which is better and more precise.
Thank you very much for your attention
Thanks for your work.
I want to use your model for training, and at the same time change the learning rate drop method and data augmentation method during training. What should I do?
hi, thanks for your work.
BAM and PAM are the attention modules which were described in the paper.
feature ectractor is modified resnet18.
However, what is CDF0? and CDFA?
Hi, @justchenhao
Traceback (most recent call last):
File "/workspace/pyroom/STANet/val.py", line 100, in <module>
val(opt)
File "/workspace/pyroom/STANet/val.py", line 64, in val
score = model.test(val=True) # run inference return confusion_matrix
File "/workspace/pyroom/STANet/models/CDFA_model.py", line 83, in test
metrics.update(self.L.detach().cpu().numpy(), pred.detach().cpu().numpy())
AttributeError: 'CDFAModel' object has no attribute 'L'
score = model.test(val=True)
self.istest = False
if opt.phase == 'test':
self.istest = True
if not self.istest:
self.L_paths = sorted(make_dataset(os.path.join(opt.dataroot, folder_L), opt.max_dataset_size))
when testing with val=True
, dataset would not load the labels self.L
so that it would raise a error.
Hi, in section 2.3 ( Implementation Details) of your paper, It said that when training on the SZTAKI dataset, the patch size is set to 784x448, which changes to 113x113 in the testing phase, however. But how can we adopt different sizes of images during training and testing?
您好,请问512x512的图片可以训练吗,我用了,但是报显存不足,我换了台机器,问题还存在;
CDFA+PAM, 512x512,batch_size=1,12G显存,应该可以啊
(epoch: 200) Overall_Acc: 0.223 Mean_IoU: 0.121 0: 0.188 1: 0.054 precision_1: 0.054 recall_1: 0.887 F1_1: 0.102
learned a lot from you, thanks. one more question. how can I label my own image dataset just like LEVIR-CD, is there any tools or code?
Hi, When I run python val.py, Error:No such file or directory: './xxxxxx_net_A.pth'. In the dir LEVIR-CDF0
, only have 'xxx_net_F.pth'files, Where is the file of 'xxxxxx_net_A.pth'?
I'm sorry to interrupt, Thank you very much !!!
@justchenhao Can you please add appropriate licence?
64GiB out of memory? what......
Hi,
I have my own implementation of precision and recall which is giving precision f1 score of 0.3 while the implementation in this repo is giving a f1 score of 0.8.
I saw that the score is implemented in a very unfamiliar way?
Could you check if your implementation is correct?
Hey, BUAAer@justchenhao
Great works! Thanks for sharing the LEVIR-CD dataset.
I wonder if all the results on the LEVIR-CD data(i.e Table 4 and Table 5
in the paper) are obtained from testing directly with input size 1024x1024
? Or on 256x256
patches and then stitch these patches into a full size image?
And would you like to build a public leadbord
with this dataset for Building Change Detection from Remote Sensing Images
? e.g. on https://paperswithcode.com/
?
可不可以先把val的测试图片切成256x256的图片呢,测试后在拼接成原图片可以吧,要不然会出现图片信息损失问题
Traceback (most recent call last):
File "F:/STANet-master/train.py", line 109, in
model = create_model(opt) # create a model given opt.model and other options
File "F:\STANet-master\models_init_.py", line 65, in create_model
instance = model(opt)
File "F:\STANet-master\models\CDFA_model.py", line 53, in init
self.isweighted = opt.weighted
AttributeError: 'Namespace' object has no attribute 'weighted'
this caused: CDFA_model.py line 53
self.isweighted = opt.weighted
no attribute 'weighted' specified in base_options.py/train_options.py
Dear author,
Thank you so much for sharing a very useful code.
i found the following error during running your val.py file, using your testing dataset with provided pretrained model:
Runtime Error: CUDA out of memory. Tried to allocate 64.00 GiB (GPU 0; 10.92 GiB total capacity; 240.92 MiB already allocated; 10.11 GiB free; 43.08 MiB cached)
My system has pytorch 1.0 with CUDA Version: 10.0
I tried to find its solution. Generally solutions are: reduce batch size, reduce the size of input images, or use latest stable version of pytorch. For testing batch size is already 1, I reduced the size of input images upto 64x64. and also install new version of pytorch in anaconda environment with "conda install pytorch torchvision cudatoolkit=10.0 -c pytorch". But i am unable to fix this problem.
How can I fix this issue?
Kindly help. Thank you!
Hi. I can't reproduce the results in the paper. I wonder if there is a problem with my training strategy or authors use selected slice images for training rather than slice images.
And I got a question that u use two independent optimizers. Is this for learning rate setting considerations or for faster convergence?
请问有人出现过在跑第某个epoch的过程中出现程序被挂起,GPU利用率为0的情况嘛
您好,我想问问关于loss函数,我在CDFA_model看到
self.L_s[self.L_s == 1] = -1 # change
self.L_s[self.L_s == 0] = 1 # no change
在loss.py里看到
class BCL(nn.Module):
"""
batch-balanced contrastive loss
no-change,1
change,-1
"""
def __init__(self, margin=2.0):
super(BCL, self).__init__()
self.margin = margin
def forward(self, distance, label):
label[label == 255] = 1
mask = (label != 255).float()
distance = distance * mask
pos_num = torch.sum((label == 1).float()) + 0.0001
neg_num = torch.sum((label == -1).float()) + 0.0001
loss_1 = torch.sum((1 + label) / 2 * torch.pow(distance, 2)) / pos_num
loss_2 = torch.sum((1 - label) / 2 *
torch.pow(torch.clamp(self.margin - distance, min=0.0), 2)
) / neg_num
loss = loss_1 + loss_2
return loss
为什么要这样写啊,或者说不改变label的值,就是原来的0和1,loss函数要怎么修改,margin的值怎么写;
还有loss函数forward下面第一句label[label == 255] = 1是不是没有用啊;
mask = (label != 255).float()
distance = distance * mask
这个好像也没用
@justchenhao
class BCL(nn.Module):
"""
batch-balanced contrastive loss
no-change,1
change,-1
"""
def __init__(self, margin=2.0):
super(BCL, self).__init__()
self.margin = margin
def forward(self, distance, label):
label[label==255] = 1
mask = (label != 255).float()
distance = distance * mask
pos_num = torch.sum((label==1).float())+0.0001
neg_num = torch.sum((label==-1).float())+0.0001
loss_1 = torch.sum((1+label) / 2 * torch.pow(distance, 2)) /pos_num
loss_2 = torch.sum((1-label) / 2 *
torch.pow(torch.clamp(self.margin - distance, min=0.0), 2)
) / neg_num
loss = loss_1 + loss_2
return loss
In this code why are you doing label[label==255] = 1
, because label is already transformed in -1 and 1. The mask
tensor below it will also be always completely 1.
whether you have tested this?
我使用了您提供的Pre-train Module,进行测试,测试结果F1为0.881,而论文中F1为87.3。请问您是取了多次的平均值吗?
hello!
请问如何解决呢,谢谢
are you upload your codes?
dataset [ChangeDetectionDataset] was created
using mynet3 backbone
arch: mynet3
in_c: 3
ds: 1
model [CDFAModel] was created
loading the model from ./checkpoints/LEVIR-CDFA0/latest_net_F.pth
loading the model from ./checkpoints/LEVIR-CDFA0/latest_net_A.pth
Traceback (most recent call last):
File "test.py", line 113, in
val(opt)
File "test.py", line 47, in val
model.setup(opt) # regular setup: load and print networks; create schedulers
File "/data2/zhouyuan/CHANGE_PROJECT/STANet-master/STANet-master/models/base_model.py", line 119, in setup
self.load_networks(load_suffix)
File "/data2/zhouyuan/CHANGE_PROJECT/STANet-master/STANet-master/models/base_model.py", line 243, in load_networks
self.__patch_instance_norm_state_dict(state_dict, net, key.split('.'))
File "/data2/zhouyuan/CHANGE_PROJECT/STANet-master/STANet-master/models/base_model.py", line 213, in __patch_instance_norm_state_dict
self.__patch_instance_norm_state_dict(state_dict, getattr(module, key), keys, i + 1)
File "/data2/zhouyuan/CHANGE_PROJECT/STANet-master/STANet-master/models/base_model.py", line 213, in __patch_instance_norm_state_dict
self.__patch_instance_norm_state_dict(state_dict, getattr(module, key), keys, i + 1)
File "/data2/zhouyuan/anaconda3.5/envs/torch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 539, in getattr
type(self).name, name))
AttributeError: 'PAM' object has no attribute 'query_conv'
Thanks for sharing the code, I have a question. How to crop the val data set to a size of 256×256? Is there a detailed code? Can you share it, thank you!
tmp = np.array(Image.open(L_path), dtype=np.uint32)/255
L_img = Image.fromarray(tmp)
transform_L = get_transform(self.opt, transform_params, method=Image.NEAREST, normalize=False,
test=self.istest)
L = transform_L(L_img)
why L is not same as A and B, and it is an error:
TypeError: Cannot handle this data type
Thankyou for your answers.
Traceback (most recent call last):
File "./train.py", line 109, in
model = create_model(opt) # create a model given opt.model and other options
File "/home/louxiantuo/STANet/models/init.py", line 65, in create_model
instance = model(opt)
File "/home/louxiantuo/STANet/models/CDFA_model.py", line 53, in init
self.isweighted = opt.weighted
AttributeError: 'Namespace' object has no attribute 'weighted'
What should I do?
I use this repo to trian my own dataset.
I set the num_class = 2(change/unchange).
model converged normally.
but the precision_1/recall_1/F1_1 does not change, always 0.
val_log.txt:
(epoch: 1) Overall_Acc: 0.971 Mean_IoU: 0.486 0: 0.971 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
================ val acc (Sun Jun 28 15:31:51 2020) ================
(epoch: 2) Overall_Acc: 0.977 Mean_IoU: 0.489 0: 0.977 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
================ val acc (Sun Jun 28 15:32:57 2020) ================
(epoch: 3) Overall_Acc: 0.979 Mean_IoU: 0.490 0: 0.979 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
================ val acc (Sun Jun 28 15:34:01 2020) ================
(epoch: 4) Overall_Acc: 0.983 Mean_IoU: 0.491 0: 0.983 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
================ val acc (Sun Jun 28 15:35:06 2020) ================
(epoch: 5) Overall_Acc: 0.987 Mean_IoU: 0.494 0: 0.987 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
================ val acc (Sun Jun 28 15:36:10 2020) ================
(epoch: 6) Overall_Acc: 0.990 Mean_IoU: 0.495 0: 0.990 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
================ val acc (Sun Jun 28 15:37:16 2020) ================
(epoch: 7) Overall_Acc: 0.970 Mean_IoU: 0.485 0: 0.970 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
================ val acc (Sun Jun 28 15:38:19 2020) ================
(epoch: 8) Overall_Acc: 0.986 Mean_IoU: 0.493 0: 0.986 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
================ val acc (Sun Jun 28 15:39:24 2020) ================
(epoch: 9) Overall_Acc: 0.978 Mean_IoU: 0.489 0: 0.978 1: 0.000 precision_1: 0.000 recall_1: 0.000 F1_1: 0.000
my gt mask is binary image, 0 or 1.
Thank you for maintaining this code. Your code and paper has brought us a lot of inspiration for our future work.
However, we also found some problems in the latest code. For example, the code can't be trained with multiple GPUs. And in the new version, we can't use visdom to preview the real-time detection results, because the image folder under the checkpoint folder is always blank, while other line graphs work normally.
We are looking forward to your early reply and wish you all the best in your future research.
希望可以得到您的指点!谢谢!
Setting up a new session...
create web directory ./checkpoints\LEVIR-CDF0\web...
(epoch: 1, iters: 200, time: 0.033, data: 4.480) f: 1.110
(epoch: 1, iters: 400, time: 0.048, data: 1.109) f: 0.909
Traceback (most recent call last):
File "./train.py", line 148, in
visualizer.plot_current_losses(epoch, float(epoch_iter) / dataset_size, losses)
File "D:\xunleiDownload\STANet-master\util\visualizer.py", line 212, in plot_current_losses
win=self.display_id)
File "D:\Software\anaconda3\envs\pytorch_gpu\lib\site-packages\visdom_init_.py", line 389, in wrapped_f
return f(*args, **kwargs)
File "D:\Software\anaconda3\envs\pytorch_gpu\lib\site-packages\visdom_init_.py", line 1694, in line
assert X.shape == Y.shape, 'X and Y should be the same shape'
AssertionError: X and Y should be the same shape
Please help me what should I do next.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.