Giter Club home page Giter Club logo

zoomnet's Introduction

(CVPR 2022) Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection

license: mit LAST COMMIT ISSUES STARS ARXIV PAPER ARXIV PAPER

@inproceedings{ZoomNet-CVPR2022,
	title     = {Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection},
	author    = {Pang, Youwei and Zhao, Xiaoqi and Xiang, Tian-Zhu and Zhang, Lihe and Lu, Huchuan},
	booktitle = CVPR,
	year      = {2022}
}

Extensions to the conference version can be found: https://github.com/lartpang/ZoomNeXt.

Changelog

  • 2022-3-16
    • Add the link of the method prediction maps of Table 1 in our paper.
  • 2022-03-08
    • Add the link of arxiv version.
  • 2022-03-07
    • Add the link of paper.
  • 2022-03-05:
    • Update weights and results links.
    • Fixed some bugs.
    • Update dataset links.
    • Update bibtex info.
  • 2022-03-04:
    • Initialize the repository.
    • Add the model and configuration file for SOD.

Usage

Dependencies

Some core dependencies:

  • timm == 0.4.12
  • torch == 1.8.1
  • pysodmetrics == 1.2.4 # for evaluating results

More details can be found in <./requirements.txt>

Datasets

More details can be found at:

Training

You can use our default configuration, like this:

$ python main.py --model-name=ZoomNet --config=configs/zoomnet/zoomnet.py --datasets-info ./configs/_base_/dataset/dataset_configs.json --info demo

or use our launcher script to start the one command in commands.txt on GPU 1:

$ python tools/run_it.py --interpreter 'abs_path' --cmd-pool tools/commands.txt  --gpu-pool 1 --verbose --max-workers 1

If you want to launch multiple commands, you can use it like this:

  1. Add your commands into the tools/commands.txt.
  2. python tools/run_it.py --interpreter 'abs_path' --cmd-pool tools/commands.txt --gpu-pool <gpu indices> --verbose --max-workers max_workers

NOTE:

  • abs_path: the absolute path of your python interpreter
  • max_workers: the maximum number of tasks to start simultaneously.

Testing

Task Weights Results
COD GitHub Release Link GitHub Release Link
SOD GitHub Release Link GitHub Release Link

For ease of use, we create a test.py script and a use case in the form of a shell script test.sh.

$ sudo chmod +x ./test.sh
$ ./test.sh 0  # on gpu 0

Method Comparisons

Paper Details

Method Detials

Comparison

Camouflaged Object Detection

Salient Object Detection

zoomnet's People

Contributors

arthurryan0803 avatar lartpang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

zoomnet's Issues

用COD10K-v3的测试集,指标差距大

你好,我用COD10K-v3的测试集进行测试的时候,指标差距很大,用NC4K测试的指标是正常的。请问是什么原因?
“COD10K-V3”:
{'Smeasure': '0.851', 'wFmeasure': '0.369', 'MAE': '0.082', 'adpEm': '0.863', 'meanEm': '0.875', 'maxEm': '0.885', 'adpFm': '0.375', 'meanFm': '0.388', 'maxFm': '0.395'}
"NC4K":
{'Smeasure': '0.853', 'wFmeasure': '0.784', 'MAE': '0.043', 'adpEm': '0.907', 'meanEm': '0.896', 'maxEm': '0.912', 'adpFm': '0.814', 'meanFm': '0.818', 'maxFm': '0.828'}
是因为没有使用测试集中的背景图片以及非伪装目标的图片吗?

如何指定在其他显卡上跑实验?

我在代码里指定了用别的显卡运行
然后输入这条指令
python main.py --model-name=ZoomNet --config=configs/zoomnet/zoomnet.py --datasets-info ./configs/base/dataset/dataset_configs.json --info demo

结果发现它还是会占用卡0的一点点显存,这是怎么回事?我想要全部搬运到其他显卡上跑实验,请问要怎么操作?

关于训练数据集处理的问题

您好!请问一下,就是关于训练数据集,是要把camo中GT部分删除剩下和image对应的1000张、挑出cod10k3040张原始图和对应的gt,然后合在一起吗?

网络参数

请问Zoomnet的整体网络参数是多少M呢

论文指标有问题

你好,根据您提供的checkpoint,我发现您在论文中report的似乎是max E_measure,但是其他论文报的应该是mean E_measure,这似乎是不是有点不合理

数据集下载与配置

您好,我最近在做复现工作,数据集这块一直也没弄好。因为我相对来说还是小白,希望你们可以提供一些复现步骤的信息(数据集配置)。十分感谢

关于PR曲线

想知道文中关于pr曲线和F曲线的绘制方法
我使用 sklearn.metrics 的 precision_recall_curve 函数
将数据集每张图的像素信息保存为 N*1 维数组再送入此函数中,但是对大型数据集来说速度非常慢
请问能否告知您的方法

Extra scale

Hi. I'm impressed by your awesome zoomnet method. And I have a question about the code.
On sod_zoomnet config, there's a config name ms.exta_size. And the config is used on baseSODdataset. And the annotation explain the value as "extra-scale for multi-scale learning". But I can't find the usage of these extra-scale information.
Because the model gets the data consist of only 1.5, 1.0, 0.5.
Therefore, could you explain the details and usage of extra-scale size? Thank you.

关于 multi-scale training 的问题

您好,在您的dataset代码中,multi-scale方法如下图:
image
image

这是否表示,不同的scale是直接从原始图像分辨率直接resize得到的。也就是说,对于1.5的scale,网络的输入分辨率实际是384*1.5=576,也即是您的网络使用了576*576的大分辨率训练。

I encountered the following error when running, what is going on?

I encountered the following error when running, what is going on?

Traceback (most recent call last):
File "main.py", line 355, in
main()
File "main.py", line 340, in main
model_ema = training(model=model, cfg=cfg)
File "main.py", line 204, in training
probs, loss, loss_str = model(
File "/home/nls1/anaconda3/envs/zjb4/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/zjb/camo/ZoomNet-main/methods/module/base_model.py", line 19, in forward
results = self.train_forward(*args, **kwargs)
File "/data/zjb/camo/ZoomNet-main/methods/zoomnet/zoomnet.py", line 224, in train_forward
output = self.body(
File "/data/zjb/camo/ZoomNet-main/methods/zoomnet/zoomnet.py", line 199, in body
l_trans_feats = self.encoder_translayer(l_scale)
File "/data/zjb/camo/ZoomNet-main/methods/zoomnet/zoomnet.py", line 195, in encoder_translayer
trans_feats = self.translayer(en_feats)
File "/home/nls1/anaconda3/envs/zjb4/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/zjb/camo/ZoomNet-main/methods/zoomnet/zoomnet.py", line 47, in forward
assert isinstance(xs, (tuple, list))
AssertionError

复现问题

您好!我是一个小白,最近我在进行复现模型的工作,但是我遇到了一个解决不了的报错:
C:\code\pythonProject\Scripts\python.exe C:/code/pythonProject/ZoomNet-main/main.py
C:\code\pythonProject\henv\Lib\site-packages\numpy_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\code\pythonProject\henv\Lib\site-packages\numpy.libs\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll
C:\code\pythonProject\henv\Lib\site-packages\numpy.libs\libopenblas64__v0.3.21-gcc_10_3_0.dll
warnings.warn("loaded more than 1 DLL from .libs:"
Traceback (most recent call last):
File "C:\code\pythonProject\ZoomNet-main\main.py", line 355, in
main()
File "C:\code\pythonProject\ZoomNet-main\main.py", line 304, in main
with open(cfg.path.cfg_copy, encoding="utf-8", mode="w") as f:
OSError: [Errno 22] Invalid argument: 'C:\code\pythonProject\ZoomNet-main\output\ZoomNet_BS8_LR0.05_E40_H384_W384_OPMsgd_OPGMfinetune_SCf3_AMP\cfg_2023-04-23 00:50:04.555687.py'

Process finished with exit code 1

请问一下,这个是什么问题呢?

关于辅助尺度的问题

在您的论文中,您提到,辅助尺度的图像是由主尺度的图像rescale得到的,然而您的代码是通过rescale原图像来获得1.5,1,0.5这三个尺度的图像的。如果按照您原文叙述,下采样得到主尺度然后上采样得到的1.5倍尺度的图像,这损失的细节要比直接对原图像下采样多。这样感觉1.5倍的尺度并不是辅助尺度了,而是主尺度。很感激您能解释一下我的疑惑,谢谢!
image

image

How to draw figure 1

Thank you for your nice work. I am a novice, I would like to ask how to draw the figure 1 of the paper.

Reproduce the results

Hi,
Thanks for sharing the code. I am interested in your paper.
I followed your instructions and ran the code with the default setting, but I cannot reproduce the results you proposed in the paper.
The following are the results I reproduced:
CHAMELEON: 'MAE': 0.032, CAMO: 'MAE': 0.075, COD10K: 'MAE': 0.033
I am wondering how to reproduce your results.
Thank you.

关于HMU模块中论文和代码不一致

您好,论文中阐述HMU的过程是讲每次处理分裂成三组的第一个,也就是g'¹与下一组g相加,但是代码里的操作是用g'²相加,最后α的产生也与论文有出入,请问具体是参照哪一个,谢谢

代码问题

想请问一下在HMU模块上,文章中描述的是每个group的第一个去跟下一个层融合,而对应代码里取的是outs[group_id-1][1],但是这里的应该为0才跟原文的意**对应吧。
for group_id in range(1, self.num_groups - 1):
branch_out = self.interact[str(group_id)](torch.cat([xs[group_id], outs[group_id - 1][1]], dim=1))
outs.append(branch_out.chunk(3, dim=1))
以及包括后面这两部分的内容, o[0]这里的0是否应该为1才对呢?
out = torch.cat([o[0] for o in outs], dim=1)
gate = self.gate_genator(torch.cat([o[-1] for o in outs], dim=1))

Cannot find "py_sod_metrics" module

I noticed that you use the "py_sod_metrics" module in the file "utils/recorder/metric_caller.py", but I can't install it in the pip. Can you give me the source code or installation package of this module?

数据集或训练配置

你好,我有一些复现的疑问,我采用COD10k中camouflaged的3040张数据和CAMO的1000张数据进行训练40轮次,训练测试结果与下载训练好的测试模型对比差了好几个点,是训练配置或是训练的数据集存在问题呢?

模型加载问题

你好,我想使用你们已经训练好的模型继续训练几轮epoch的时候,进行到这一步的时候
if cfg.resume_from:
params_in_checkpoint = io.load_specific_params(
load_path=cfg.resume_from, names=["model", "optimizer", "scaler", "start_epoch"]
)
报出了这个错误
"There is not 'net_state' in ./output/ForSharing/cod_zoomnet_r50_bs8_e40_2022-03-04.pth: ['shared_encoder.conv1.weight',

请问这个该如何解决呢?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.