qqlu / entity Goto Github PK
View Code? Open in Web Editor NEWEntitySeg Toolbox: Towards Open-World and High-Quality Image Segmentation
License: Other
EntitySeg Toolbox: Towards Open-World and High-Quality Image Segmentation
License: Other
where can i find code of the paper?
Hello! Have you tried to refine the mask which is more than one class at same time, such as 3 classes or 4 classes? And how about the result? Thank you very much.
When I run "python3 entity_to_json.py", the following problems occur:
Traceback (most recent call last):
File "entity_to_json.py", line 23, in
catid_map = {category['id']: [cid, category["isthing"], category["name"], category["supercategory"]] for cid, category in enumerate(categories_list)}
File "entity_to_json.py", line 23, in
catid_map = {category['id']: [cid, category["isthing"], category["name"], category["supercategory"]] for cid, category in enumerate(categories_list)}
KeyError: 'supercategory'
I find that the problem occurs in categories_list, and there is no "supercategory" key in categories_list(from detectron2.data.datasets.builtin_meta import COCO_CATEGORIES) .
I am trying to training with CUDA11.1 on 2080Ti x4.
The training crashed after ~600 iteration with
[08/28 15:43:34] fvcore.common.checkpoint WARNING: The checkpoint state_dict contains keys that are not used by the model:
[35mfc1000.{bias, weight}�[0m
[08/28 15:43:34] detectron2.engine.train_loop INFO: Starting training from iteration 0
[08/28 15:56:53] detectron2.engine.train_loop ERROR: Exception during training:
Traceback (most recent call last):
File "/data/detectron2/detectron2/engine/train_loop.py", line 149, in train
self.run_step()
File "/data/detectron2/detectron2/engine/defaults.py", line 495, in run_step
self._trainer.run_step()
File "/data/detectron2/detectron2/engine/train_loop.py", line 273, in run_step
loss_dict = self.model(data)
File "/opt/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 799, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/opt/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/data/detectron2/projects/EntitySeg/entityseg/arch.py", line 103, in forward
num_instances = int(torch.max(instance_map)+1)
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
[08/28 15:56:53] detectron2.engine.hooks INFO: Overall training speed: 653 iterations in 0:13:04 (1.2021 s / it)
[08/28 15:56:53] detectron2.engine.hooks INFO: Total training time: 0:13:05 (0:00:00 on hooks)
When I train with 2 GPUs, I have the same problem in facebookresearch/detectron2#4191.
Thus, after reading your README.MD, I want to know how to set the "find_unused_parameters=True" in distributed training of my own detectron2.
I have tried to set the "find_unused_parameters=True" in the detectron2/engine/defaults.py
Thus, I want to ask you whether I change it correctly.
I am looking forward to your response.
Thanks for your work。Can the Cropformer-Swin model be converted to tensorRT? How can I improve the speed of predict? I am currently testing 50 images using NVIDIA GeForce RTX 3090, which takes 15 minutes.
Hello, you‘ve mentioned at the beginning of your abstract that the traditional approach have the problem of balancing computational cost and computational time. So I want to know how much graphics card memory and computing time your method needs when conducting 2K or 6K images?And what kind of graphics card we need to run this code? Thank you.
Hi, why do I get this error when I run the coco instance_2017 dataset with the source program?
This error is as follows
Traceback (most recent call last):
File "/home/hndx/detectron2-main/detectron2/projects/EntitySeg/train_net.py", line 81, in
args=(args,),
File "/home/hndx/detectron2-main/detectron2/engine/launch.py", line 82, in launch
main_func(*args)
File "/home/hndx/detectron2-main/detectron2/projects/EntitySeg/train_net.py", line 67, in main
trainer = Trainer(cfg)
File "/home/hndx/detectron2-main/detectron2/engine/defaults.py", line 378, in init
data_loader = self.build_train_loader(cfg)
File "/home/hndx/detectron2-main/detectron2/projects/EntitySeg/train_net.py", line 33, in build_train_loader
return build_detection_train_loader(cfg, mapper)
File "/home/hndx/detectron2-main/detectron2/config/config.py", line 207, in wrapped
explicit_args = _get_args_from_config(from_config, *args, **kwargs)
File "/home/hndx/detectron2-main/detectron2/config/config.py", line 245, in _get_args_from_config
ret = from_config_func(*args, **kwargs)
File "/home/hndx/detectron2-main/detectron2/data/build.py", line 350, in _train_loader_from_config
proposal_files=cfg.DATASETS.PROPOSAL_FILES_TRAIN if cfg.MODEL.LOAD_PROPOSALS else None,
File "/home/hndx/detectron2-main/detectron2/data/build.py", line 278, in get_detection_dataset_dicts
assert len(dataset_dicts), "No valid data found in {}.".format(",".join(names))
AssertionError: No valid data found in coco_2017_train_entity.
@qqlu 哈喽,您好,CropFormer训练代码有实现吗?是用的mask2former的代码吗?
Hi, I have tried to realize your method(High Quality Segmentation for Ultra High-resolution Images), it works well. But do you have met the problem that the wrong mask segmentation happens when the RGB-image area is similar to the real label area color. For example , I want to segment the sky area, but the white wall also to be thinked as sky.
Hi! I wonder how many and what kinds of GPUs did you use in UniGS? How long does the training process take? Thanks!
Hi,I am interested in your this paper, but I have some question. Can you explain the pseudo-label dataset generation stage? How to calculated the score of any prediction and (constant)score threshold δ is meaning box confidence threshold?
作者您好:
很高兴看到您的UniGS论文被cvpr 2024接收,我对您这篇工作比较好奇和感兴趣,特别生成和分割统一表示的部分以及之前您做过的高质量分割,请问啥时候可以开源下代码~
FileNotFoundError: [Errno 2] No such file or directory: 'D:/Code/Project/Carvana_dataset/testimage\Pic_2023_03_18_091334_5_seg.png'
Why do you use an imperfect library detectron2?
Could you explain the pretrained models form this page?
Were all of these models pretrained using the COCO dataset and the numbers are evaluation numbers on the EntitySeg-{semantic, instance, panoptic, entity} datasets? Or are they just starting with COCO pretrained weights, and then fine-tuned on the EntitySeg dataset and the evaluation numbers are from some other dataset?
When i print the ori_scores and scores in demo_result_and_vis.py, the scores is is not sorted from high to low. The reason is mask rescoring. But score in scores which higher than args.confidence_threshold is discarded in visualization.
Where can I find the code of the CA-SSL?
Hello guys,
I've read "High Quality Segmentation for Ultra High-resolution Images" recently. Great work!
I am very excited about this paper.
When do you plan to release code for this paper?
Thanks in advance.
Hello!
This work of AIMS is intersting.
When do you plan to release the code of the paper "AIMS: All-Inclusive Multi-Level Segmentation", please?
Hi, I am getting the following error when running this coarse partition network with train2017 dataset
ERROR [08/05 15:18:06 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
File "/home/hndx/detectron2-main/detectron2/engine/train_loop.py", line 149, in train
self.run_step()
File "/home/hndx/detectron2-main/detectron2/engine/defaults.py", line 494, in run_step
self._trainer.run_step()
File "/home/hndx/detectron2-main/detectron2/engine/train_loop.py", line 268, in run_step
data = next(self._data_loader_iter)
File "/home/hndx/detectron2-main/detectron2/data/common.py", line 234, in iter
for d in self.dataset:
File "/home/hndx/anaconda3/envs/llz0/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/home/hndx/anaconda3/envs/llz0/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/home/hndx/anaconda3/envs/llz0/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/home/hndx/anaconda3/envs/llz0/lib/python3.7/site-packages/torch/_utils.py", line 434, in reraise
raise exception
AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/hndx/anaconda3/envs/llz0/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/hndx/anaconda3/envs/llz0/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 32, in fetch
data.append(next(self.dataset_iter))
File "/home/hndx/detectron2-main/detectron2/data/common.py", line 201, in iter
yield self.dataset[idx]
File "/home/hndx/detectron2-main/detectron2/data/common.py", line 90, in getitem
data = self._map_func(self._dataset[cur_idx])
File "/home/hndx/detectron2-main/detectron2/utils/serialize.py", line 26, in call
return self._obj(*args, **kwargs)
File "/home/hndx/detectron2-main/detectron2/projects/EntitySeg/entityseg/data/dataset_mapper.py", line 197, in call
instances.instanceid = instance_id_list
File "/home/hndx/detectron2-main/detectron2/structures/instances.py", line 66, in setattr
self.set(name, val)
File "/home/hndx/detectron2-main/detectron2/structures/instances.py", line 84, in set
), "Adding a field of length {} to a Instances of length {}".format(data_len, len(self)) ##lizhi long
I run test.py but i met this error, miss "_seg.png" file
python test.py --dir ./data/DUTS-TE --model ./model_10000 --output ./output --clear
/home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: /home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/torchvision/image.so: undefined symbol: _ZNK2at10TensorBase21__dispatch_contiguousEN3c1012MemoryFormatE
warn(f"Failed to load image Python extension: {e}")
before_Parser_time: 1659253874.6776164
Hyperparameters: {'dir': './data/DUTS-TE', 'model': './model_10000', 'output': './output', 'global_only': False, 'L': 900, 'stride': 450, 'clear': True, 'ade': False}
ASPP_4level
12 images found
before_for_time: 1659253881.0989463 ; before_for_time - before_Parser_time: 6.421329975128174
Traceback (most recent call last):
File "test.py", line 106, in
for im, seg, gt, name, crm_data in progressbar.progressbar(val_loader):
File "/home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/progressbar/shortcuts.py", line 10, in progressbar
for result in progressbar(iterator):
File "/home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/progressbar/bar.py", line 547, in next
value = next(self._iterable)
File "/home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 561, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/hoang/Desktop/luanvan/implicitmodel/4cham/Entity/High-Quality-Segmention/dataset/offline_dataset_crm_pad32.py", line 138, in getitem
im, seg, gt = self.load_tuple(self.im_list[idx])
File "/home/hoang/Desktop/luanvan/implicitmodel/4cham/Entity/High-Quality-Segmention/dataset/offline_dataset_crm_pad32.py", line 110, in load_tuple
seg = Image.open(im[:-7]+'_seg.png').convert('L')
File "/home/hoang/anaconda3/envs/tranSOD/lib/python3.6/site-packages/PIL/Image.py", line 2975, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: './data/DUTS-TE/sun_abzsyxfgntlvd_seg.png'
Hi,
The results of "Fine-Grained Entity Segmentation" look very exciting! When do you plan to release the data and code for this paper?
Thanks!
It seems that entity loads gt_bitmasks
from .npz
files built from PNG files, instead of using gt_masks
from the RLEs in the instances_*.json
.
Is there any particular reason for doing so?
If it has to be like this, it would be favorable to provide scripts to convert coco RLE jsons into PNG files which could then be used in make_entity_mask.py
Thanks for your great work "Open-World Entity Segmentation". Could you provide the evaluation code on ADE-20K dataset, when using models trained on COCO?
Thanks for your great work. I have a question that how the panoptic results of PanoFCN or DETR transfered to the entity segmentation results?
hello, thanks for your great work High Quality Segmentation.
I want to ask the following 2 questions
Hi,thank you for your contributions to entity segmentation task. When I train the model followed your steps, I encountered the following problems: FloatingPointError: Loss became infinite or NaN at iteration=20! . I am looking forward to you reply.
results of pretrained model mit_b5_1x.pth on COCO val set are not good.Here is my command:
python projects/EntitySeg/train_net.py --config-file projects/EntitySeg/configs/entity_mit_b5_1x.yaml --num-gpus 1 --eval-only MODEL.WEIGHTS data/models/mit_b5_1x.pth MODEL.CONDINST.MASK_BRANCH.USE_MASK_RESCORE "True"
and the results:
[08/11 16:11:11 d2.engine.defaults]: Evaluation results for coco_2017_val_entity in csv format: [08/11 16:11:11 d2.evaluation.testing]: copypaste: Task: bbox [08/11 16:11:11 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl [08/11 16:11:11 d2.evaluation.testing]: copypaste: 0.0010,0.0019,0.0008,0.0004,0.0014,0.0013 [08/11 16:11:11 d2.evaluation.testing]: copypaste: Task: segm [08/11 16:11:11 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl [08/11 16:11:11 d2.evaluation.testing]: copypaste: 0.0006,0.0015,0.0005,0.0002,0.0008,0.0009
Hi there, thanks for your work! It looks very interesting and i would like to test it out, however I keep running into the same problem when testing the visualization script and wondering if you could help me out.
Steps I did:
Any help is greatly appreciated. Thank you!
First of all, many thanks to the authors for open sourcing such a great project. I was a little confused about the image format conversion.
Why use 'BGR' to input image in demo_crop, set cfg.INPUT.FORMAT = 'RGB', and then flip image to 'RGB' before the image is fed into the network? Doesn't this cause confusion, or is there something special going on.
Hi,
Thanks for releasing such a great dataset! I am trying to replicate the result using the released weights on Huggingface, but found that some config files are missing, such as this one. Could you upload these files?
Thanks
Thanks for your work. I met a problem when training coco panoptic model by using multi GPUs.
My command is python3 projects/EntitySeg/train_net.py --config-file projects/EntitySeg/configs/entity_swin_t_1x.yaml --num-gpus 4 OUTPUT_DIR entity_model
. Below is my error:
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`; (2) making sure all `forward` function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
Any help is greatly appreciated. Thank you!
I am running EntitySegRLE and there seems to be some files missing.
I am getting a ModuleNotFoundError: No module named 'entityseg.arch_rle_aug' error.
Additionally, in the config files, there seems to be only a few config files compared to the available models. How do I get the relevant files? Thank you!!
Great work! I am using the mask2former entity seg model and after extracting the masks and their confidences using prediction.score, it seems that all of the values are 0.7. Any idea why this might be happening? Thank you!
Hi,
When do you plan to release the data and code of the paper "Fine-Grained Entity Segmentation"?
Thanks!
Hello, thanks for your contribution. When I used make_entity_mask.py
to genere my own annotation, I found a problem in generating entity mask. When a small object is contained by a large object, if the small object is annotated first, the annotation of the small object will be covered by the annotation of the large object later. For example, in the picture below, the object in the red frame is covered
By the way, my task is open world instance segmentation, I think this problem will not be met in the panoptic segmentation because the different annotation format. My method is to annotate objects according to their area from large to small, so that small objects will not be covered.
It seems current entity_evaluation.py can only calculate entity AP for coco datasets only. Yet in your paper, there are results for other datasets like ade20k and cityscapes. Could you please provide code for the evaluation of those datasets? Thank you very much.
'torch.cuda.empty_cache()' seems don`t work after process one chunk image, I get an RuntimeError: CUDA out of memory.
你好,请问模型公开了吗?点击进去,没有可以下载的模型。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.