hustvl / tevit Goto Github PK
View Code? Open in Web Editor NEWTemporally Efficient Vision Transformer for Video Instance Segmentation, CVPR 2022, Oral
Home Page: https://arxiv.org/abs/2204.08412
License: MIT License
Temporally Efficient Vision Transformer for Video Instance Segmentation, CVPR 2022, Oral
Home Page: https://arxiv.org/abs/2204.08412
License: MIT License
Thanks for this work.
Can you please share how we can test the model on one image or video.
Thanks in advance
Wonderful work!
We're very interested in your work. VIS is a future key development direction in mmtracking. We'll appreciate it if you can create a PR about this work in mmtracking.
Thanks for your excellent work. Do you have any plans to release the code of the online reference recently?
Hi,
Thanks for uploading the code and for a great paper. I have a few questions about the method as I've been reading the paper but found it difficult to understand from the codebase its implementation.
DynConv
layer for all N_H
STQI heads. Is this DynConv
layer within each STQI head the same as in QueryInst? i.e. q_t <-- DynConv_box(p_box, q_t-1)
. where p_box
are ROI-pooled instance featuresDynamic Conv
per head. In QueryInst
there are both dynamic mask and dynamic box layers for each stage. Can you confirm there is only DynConv_box
in STQI?MsgShiftT
or Swin
are multi-scale. How are the multi-resolution features dealt with in DynConv_box
or DynConv_mask
. I can't find this information in the manuscript. Do you make predictions for every scale like in an FPN network?N_H
STQI-heads replace the 6 stages you might have in QueryInst
?Many thanks!
Traceback (most recent call last):
File "./tools/train.py", line 17, in
from mmdet.apis import init_random_seed, set_random_seed, train_detector
File "/home/hss/TeViT-main/mmdet/apis/init.py", line 2, in
from .inference import (async_inference_detector, inference_detector,
File "/home/hss/TeViT-main/mmdet/apis/inference.py", line 12, in
from mmdet.datasets import replace_ImageToTensor
File "/home/hss/TeViT-main/mmdet/datasets/init.py", line 18, in
Traceback (most recent call last):
File "./tools/train.py", line 17, in
from .youtubevis import YoutubeVISDataset
File "/home/hss/TeViT-main/mmdet/datasets/youtubevis.py", line 9, in
from pycocotools.ytvos import YTVOS
ModuleNotFoundError: No module named 'pycocotools.ytvos'
from mmdet.apis import init_random_seed, set_random_seed, train_detector
File "/home/hss/TeViT-main/mmdet/apis/init.py", line 2, in
from .inference import (async_inference_detector, inference_detector,
File "/home/hss/TeViT-main/mmdet/apis/inference.py", line 12, in
from mmdet.datasets import replace_ImageToTensor
File "/home/hss/TeViT-main/mmdet/datasets/init.py", line 18, in
from .youtubevis import YoutubeVISDataset
File "/home/hss/TeViT-main/mmdet/datasets/youtubevis.py", line 9, in
from pycocotools.ytvos import YTVOS
Hello,
Thank you for publishing this awesome work. I'm struggling with this error while testing tevit on one image.
result = inference_detector(model, img) File "/mmdetection/tevit/mmdet/apis/inference.py", line 137, in inference_detector data['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']] TypeError: 'DataContainer' object is not iterable
Thanks in advance.
Hi, vealocia,
Maybe we need to do it first to run this code.
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI
So I think you can add it to requirements.๐ฅฐ
Thanks for your work. I'm struggling for days with this error. Can you please provide some solutions to overcome it.
(base) root@78845bc2a82a:/mmdetection/Tevit# python tools/test_vis.py configs/tevit/tevit_msgshift.py checkpoint/tevit_r50.pth
load checkpoint from local path: checkpoint/tevit_r50.pth
Traceback (most recent call last):
File "tools/test_vis.py", line 137, in
main(args)
File "tools/test_vis.py", line 54, in main
cfg_options=args.cfg_options)
File "/opt/conda/lib/python3.7/site-packages/mmdet/apis/inference.py", line 45, in init_detector
checkpoint = load_checkpoint(model, checkpoint, map_location='cpu')
File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 581, in load_checkpoint
checkpoint = _load_checkpoint(filename, map_location, logger)
File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 520, in _load_checkpoint
return CheckpointLoader.load_checkpoint(filename, map_location, logger)
File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 285, in load_checkpoint
return checkpoint_loader(filename, map_location)
File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 302, in load_from_local
checkpoint = torch.load(filename, map_location=map_location)
File "/opt/conda/lib/python3.7/site-packages/torch/serialization.py", line 585, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/opt/conda/lib/python3.7/site-packages/torch/serialization.py", line 755, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
EOFError: Ran out of input
Hi, I am trying to run this repo and i have followed all the steps mentioned in the description.
While running the inference code, I am getting the following error.
python tools/test_vis.py configs/tevit/tevit_msgshift.py checkpoints/tevit_r50.pth Traceback (most recent call last): File "tools/test_vis.py", line 130, in <module> main(args) File "tools/test_vis.py", line 53, in main cfg_options=args.cfg_options) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmdet/apis/inference.py", line 43, in init_detector model = build_detector(config.model, test_cfg=config.get('test_cfg')) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmdet/models/builder.py", line 59, in build_detector cfg, default_args=dict(train_cfg=train_cfg, test_cfg=test_cfg)) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmcv/utils/registry.py", line 212, in build return self.build_func(*args, **kwargs, registry=self) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg return build_from_cfg(cfg, registry, default_args) File "/home/quidich/.virtualenvs/tevit/lib/python3.6/site-packages/mmcv/utils/registry.py", line 45, in build_from_cfg f'{obj_type} is not in the {registry.name} registry') KeyError: 'TeViT is not in the models registry'
Kindly, could you guide me if I want to create a file that directly runs this algorithm on a video or set of images and provide an out in the form of images and json format which gets saved in some folder?
Thanks for your excellent work.
For the test of image_demo.py(or video_demo.py) in /demo, use the demo.jpg(or demo.mp4) as input, there is a problem.
Could you please provide some advice? Looking forward to your reply.
/TeViT/mmdet/apis/inference.py:50: UserWarning: Class names are not saved in the checkpoint's meta data, use COCO classes by default.
warnings.warn('Class names are not saved in the checkpoint's '
Traceback (most recent call last):
File "image_demo.py", line 65, in
main(args)
File "image_demo.py", line 35, in main
result = inference_detector(model, args.img)
File "/TeViT/mmdet/apis/inference.py", line 137, in inference_detector
data['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']]
TypeError: 'DataContainer' object is not iterable
File "video_demo.py", line 61, in
main()
File "video_demo.py", line 47, in main
result = inference_detector(model, frame)
File "/TeViT/mmdet/apis/inference.py", line 137, in inference_detector
data['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']]
TypeError: 'DataContainer' object is not iterable
Hello,
I am encountering an issue with the TeViT model and need your assistance. I am using the video_demo.py script for the inference part of the TeViT model, but when I run the script, I get the following error:
KeyError: 'TeViT is not in the mmdet::model registry. Please check whether the value of TeViT
is correct or it was registered as expected.'
Could you please help me understand the root cause of this error and how it can be fixed? I need to ensure that I have correctly added the TeViT model to the mmdet::model registry, but I am having trouble identifying what might have gone wrong.
If there are any additional details I need to add or any other corrections to the command-line script I'm using, please let me know.
Thank you!
!python video_demo.py video.mp4 config_file.py checkpoint.pth --out output.mp4 --show
Thanks for your great work. I am not familiar with the youtubevis API for bbox evaluation and want to learn about the evaluation procedure base on the provided trainset annotation. Besides the evaluation code on segmentation, can you provide a code that can store the bbox prediction result in standard format for evaluation? Thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.