gaomingqi / track-anything Goto Github PK
View Code? Open in Web Editor NEWTrack-Anything is a flexible and interactive tool for video object tracking and segmentation, based on Segment Anything, XMem, and E2FGVI.
License: MIT License
Track-Anything is a flexible and interactive tool for video object tracking and segmentation, based on Segment Anything, XMem, and E2FGVI.
License: MIT License
In the Steph Curry example, how many different manual annotations needed to be made before processing? I am attempting to process a similar video, but can not keep consistency of a person between different camera cuts
运行情况:(supple) sepplu@seppdeMac-Studio Track-Anything % python3 app.py --device cpu
Defaulting to user installation because normal site-packages is not writeable
Looking in links: https://download.openmmlab.com/mmcv/dist/cpu/torch2.0.0/index.html
Requirement already satisfied: mmcv in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (2.0.0)
Requirement already satisfied: addict in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmcv) (2.4.0)
Requirement already satisfied: mmengine>=0.2.0 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmcv) (0.7.3)
Requirement already satisfied: numpy in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmcv) (1.24.3)
Requirement already satisfied: packaging in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmcv) (23.1)
Requirement already satisfied: Pillow in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmcv) (9.5.0)
Requirement already satisfied: pyyaml in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmcv) (6.0)
Requirement already satisfied: yapf in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmcv) (0.33.0)
Requirement already satisfied: matplotlib in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmengine>=0.2.0->mmcv) (3.7.1)
Requirement already satisfied: rich in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmengine>=0.2.0->mmcv) (13.3.5)
Requirement already satisfied: termcolor in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmengine>=0.2.0->mmcv) (2.3.0)
Requirement already satisfied: opencv-python>=3 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from mmengine>=0.2.0->mmcv) (4.7.0.72)
Requirement already satisfied: tomli>=2.0.1 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from yapf->mmcv) (2.0.1)
Requirement already satisfied: contourpy>=1.0.1 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from matplotlib->mmengine>=0.2.0->mmcv) (1.0.7)
Requirement already satisfied: cycler>=0.10 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from matplotlib->mmengine>=0.2.0->mmcv) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from matplotlib->mmengine>=0.2.0->mmcv) (4.39.3)
Requirement already satisfied: kiwisolver>=1.0.1 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from matplotlib->mmengine>=0.2.0->mmcv) (1.4.4)
Requirement already satisfied: pyparsing>=2.3.1 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from matplotlib->mmengine>=0.2.0->mmcv) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from matplotlib->mmengine>=0.2.0->mmcv) (2.8.2)
Requirement already satisfied: importlib-resources>=3.2.0 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from matplotlib->mmengine>=0.2.0->mmcv) (5.12.0)
Requirement already satisfied: markdown-it-py<3.0.0,>=2.2.0 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from rich->mmengine>=0.2.0->mmcv) (2.2.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from rich->mmengine>=0.2.0->mmcv) (2.15.1)
Requirement already satisfied: zipp>=3.1.0 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from importlib-resources>=3.2.0->matplotlib->mmengine>=0.2.0->mmcv) (3.15.0)
Requirement already satisfied: mdurl~=0.1 in /Users/sepplu/Library/Python/3.9/lib/python/site-packages (from markdown-it-py<3.0.0,>=2.2.0->rich->mmengine>=0.2.0->mmcv) (0.1.2)
Requirement already satisfied: six>=1.5 in /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->mmengine>=0.2.0->mmcv) (1.15.0)
Initializing BaseSegmenter to cpu
Traceback (most recent call last):
File "/Users/sepplu/Track-Anything/app.py", line 383, in
model = TrackingAnything(SAM_checkpoint, xmem_checkpoint, e2fgvi_checkpoint,args)
File "/Users/sepplu/Track-Anything/track_anything.py", line 19, in init
self.xmem = BaseTracker(self.xmem_checkpoint, device=args.device)
File "/Users/sepplu/Track-Anything/tracker/base_tracker.py", line 32, in init
network = XMem(config, xmem_checkpoint).to(device).eval()
File "/Users/sepplu/Track-Anything/tracker/model/network.py", line 24, in init
model_weights = self.init_hyperparameters(config, model_path, map_location)
File "/Users/sepplu/Track-Anything/tracker/model/network.py", line 145, in init_hyperparameters
model_weights = torch.load(model_path, map_location=map_location)
File "/Users/sepplu/miniconda3/envs/supple/lib/python3.10/site-packages/torch/serialization.py", line 809, in load
return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
File "/Users/sepplu/miniconda3/envs/supple/lib/python3.10/site-packages/torch/serialization.py", line 1172, in _load
result = unpickler.load()
File "/Users/sepplu/miniconda3/envs/supple/lib/python3.10/site-packages/torch/serialization.py", line 1142, in persistent_load
typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
File "/Users/sepplu/miniconda3/envs/supple/lib/python3.10/site-packages/torch/serialization.py", line 1116, in load_tensor
wrap_storage=restore_location(storage, location),
File "/Users/sepplu/miniconda3/envs/supple/lib/python3.10/site-packages/torch/serialization.py", line 217, in default_restore_location
result = fn(storage, location)
File "/Users/sepplu/miniconda3/envs/supple/lib/python3.10/site-packages/torch/serialization.py", line 182, in _cuda_deserialize
device = validate_cuda_device(location)
File "/Users/sepplu/miniconda3/envs/supple/lib/python3.10/site-packages/torch/serialization.py", line 166, in validate_cuda_device
raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
When I followed the instruction to install all the requirements, I still got some problems as follow:
Initializing BaseSegmenter to cuda:0
Traceback (most recent call last):
File "E:\Sydney University\CVProject\Project\Track_Anything\Track-Anything-master\app.py", line 343, in
model = TrackingAnything(SAM_checkpoint, xmem_checkpoint, e2fgvi_checkpoint,args)
File "E:\Sydney University\CVProject\Project\Track_Anything\Track-Anything-master\track_anything.py", line 13, in init
self.samcontroler = SamControler(sam_checkpoint, args.sam_model_type, args.device)
File "E:\Sydney University\CVProject\Project\Track_Anything\Track-Anything-master\tools\interact_tools.py", line 37, in init
self.sam_controler = BaseSegmenter(SAM_checkpoint, model_type, device)
File "E:\Sydney University\CVProject\Project\Track_Anything\Track-Anything-master\tools\base_segmenter.py", line 25, in init
self.model = sam_model_registrymodel_type
File "D:\Anaconda\envs\Track_Anything\lib\site-packages\segment_anything\build_sam.py", line 15, in build_sam_vit_h
return _build_sam(
File "D:\Anaconda\envs\Track_Anything\lib\site-packages\segment_anything\build_sam.py", line 105, in _build_sam
state_dict = torch.load(f)
File "D:\Anaconda\envs\Track_Anything\lib\site-packages\torch\serialization.py", line 705, in load
with _open_zipfile_reader(opened_file) as opened_zipfile:
File "D:\Anaconda\envs\Track_Anything\lib\site-packages\torch\serialization.py", line 242, in init
super(_open_zipfile_reader, self).init(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
Is there anyone meet the same problem and solve it?
Traceback (most recent call last):
File "H:\Deepfacelab\Deepface\Track-Anything\venv\lib\site-packages\gradio\routes.py", line 395, in run_predict
output = await app.get_blocks().process_api(
File "H:\Deepfacelab\Deepface\Track-Anything\venv\lib\site-packages\gradio\blocks.py", line 1193, in process_api
result = await self.call_function(
File "H:\Deepfacelab\Deepface\Track-Anything\venv\lib\site-packages\gradio\blocks.py", line 916, in call_function
prediction = await anyio.to_thread.run_sync(
File "H:\Deepfacelab\Deepface\Track-Anything\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "H:\Deepfacelab\Deepface\Track-Anything\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "H:\Deepfacelab\Deepface\Track-Anything\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run
result = context.run(func, *args)
File "H:\Deepfacelab\Deepface\Track-Anything\app.py", line 130, in sam_refine
image=video_state["origin_images"][video_state["select_frame_number"]],
TypeError: 'NoneType' object is not subscriptable
Hi all,
First of all, I would like to thank you again for your amazing work! I have successfully used it in my application, and it performs really well! However, I have noticed a performance issue that I would like to address to further improve the tracking capabilities of the model.
When tracking ONE object, if the object disappears (entirely) from the video for a few frames and then reappears, the model seems to struggle to identify it again, without using human interactions. I understand this is a common issue. However, I am wondering if there is anything I could modify in the code to enhance the model's ability to track objects that temporarily disappear and reappear in the video.
Any suggestions or guidance would be greatly appreciated. Thank you in advance!
Best regards,
$ python app.py --device cuda:0
'mim' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "D:\remover vdieo\Track-Anything\app.py", line 159, in
def sam_refine(video_state, point_prompt, click_state, interactive_state, evt:gr.SelectData):
^^^^^^^^^^^^^
AttributeError: module 'gradio' has no attribute 'SelectData'
app.py --device cuda:0
Initializing BaseSegmenter to cuda:0
Traceback (most recent call last):
File "C:\Users\Happy\Desktop\Git\Track-Anything\app.py", line 383, in
model = TrackingAnything(SAM_checkpoint, xmem_checkpoint, e2fgvi_checkpoint,args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Happy\Desktop\Git\Track-Anything\track_anything.py", line 18, in init
self.samcontroler = SamControler(self.sam_checkpoint, args.sam_model_type, args.device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Happy\Desktop\Git\Track-Anything\tools\interact_tools.py", line 37, in init
self.sam_controler = BaseSegmenter(SAM_checkpoint, model_type, device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Happy\Desktop\Git\Track-Anything\tools\base_segmenter.py", line 26, in init
self.model.to(device=self.device)
File "C:\Users\Happy\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1145, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Happy\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
module._apply(fn)
File "C:\Users\Happy\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
module._apply(fn)
File "C:\Users\Happy\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
module._apply(fn)
File "C:\Users\Happy\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 820, in apply
param_applied = fn(param)
^^^^^^^^^
File "C:\Users\Happy\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1143, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Happy\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\cuda_init.py", line 239, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
I don't know how to fix this, can anyone help?
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.17 GiB (GPU 0; 14.84 GiB total capacity; 11.81 GiB already allocated; 1.67 GiB free; 12.22 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Still getting errors even with 16GB GPU memory
as what i questioned, i don't have Nvidia device,just mps
Hi,I am the one who Email you about my browser happen errors,yesterday. Today,I input "python app.py --device cuda:0 --sam_model_type vit_b",It works ok!Is this due to my GPU?I use 2070 8g.
Another question,if I want to know the center point which area I clicked.What can I do for it.
Hi, I have successfully installed Tracking-Anything, and it works mostly as expected.
A minor bug is that, I cannot use either cv2.imshow() or plt.show() to plot any images, the code stucks forever.
I have created a new virtual environment testing the same code with only SAM installed, and it works fine. But with Tracking-Anything installed, the same bug happens.
Could you please look into this issue?
Many thanks!
also is any ways to get the alpha mask of tracked object?
First of all this is an outstanding project.
unfortunately i could not replicate it here, since this error prevents the server to run.
python app.py --device cuda:0 -sam_model_type vit_h --port 12212
usage: app.py [-h] [--device DEVICE] [--sam_model_type SAM_MODEL_TYPE]
[--port PORT] [--debug] [--mask_save MASK_SAVE]
app.py: error: unrecognized arguments: -sam_model_type vit_h
any tip to what to do, before I try to fix it myself?
cheers
(base) root@185:~/Track-Anything# python app.py --device cuda:0 --sam_model_type vit_h --port 80
Traceback (most recent call last):
File "/root/Track-Anything/app.py", line 2, in
from demo import automask_image_app, automask_video_app, sahi_autoseg_app
File "/root/Track-Anything/demo.py", line 1, in
from metaseg import SegAutoMaskPredictor, SegManualMaskPredictor, SahiAutoSegmentation, sahi_sliced_predict
File "/root/anaconda3/lib/python3.10/site-packages/metaseg/init.py", line 7, in
from metaseg.falai_demo import falai_automask_image, falai_manuelmask_image
File "/root/anaconda3/lib/python3.10/site-packages/metaseg/falai_demo.py", line 5, in
from metaseg import SegAutoMaskPredictor, SegManualMaskPredictor
ImportError: cannot import name 'SegAutoMaskPredictor' from partially initialized module 'metaseg' (most likely due to a circular import) (/root/anaconda3/lib/python3.10/site-packages/metaseg/init.py)
Can I export the tracked object as a mask? is it built in to this? (I cant seem to find it). Also, I am no programmer!
Seems like new metaseg==0.7 has some circular dependencies
I think it's a external problem, but someone know any solution?
Nvidia/cuda drivers updated.
E:\Segment\Track-Anything>app.py
download checkpoints ......
download successfully!
download checkpoints ......
download successfully!
Initializing BaseSegmenter to cuda:4
Traceback (most recent call last):
File "E:\Segment\Track-Anything\app.py", line 200, in
model = TrackingAnything(SAM_checkpoint, xmem_checkpoint, args)
File "E:\Segment\Track-Anything\track_anything.py", line 14, in init
self.samcontroler = SamControler(sam_checkpoint, args.sam_model_type, args.device)
File "E:\Segment\Track-Anything\tools\interact_tools.py", line 37, in init
self.sam_controler = BaseSegmenter(SAM_checkpoint, model_type, device)
File "E:\Segment\Track-Anything\tools\base_segmenter.py", line 26, in init
self.model.to(device=self.device)
File "C:\Users\marco\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 990, in to
return self._apply(convert)
File "C:\Users\marco\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 642, in _apply
module._apply(fn)
File "C:\Users\marco\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 642, in _apply
module._apply(fn)
File "C:\Users\marco\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 642, in _apply
module._apply(fn)
File "C:\Users\marco\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 665, in _apply
param_applied = fn(param)
File "C:\Users\marco\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 988, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Hi,
Thank you for such an amazing and useful project! I am wondering what the best performance of the running time for each frame you got when tracking an object in a video? The reason I am asking is that I am thinking to use this powerful tool in a real-time AR application but not sure whether it could be capable of this. Thank you again for your time and effort!
Holly@HOLLY MINGW64 ~/Documents/Track-Anything (master)
$ python app.py --device cuda:0
C:\Users\Holly\AppData\Local\Programs\Python\Python310\lib\site-packages\mmcv_init_.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
warnings.warn(
Initializing BaseSegmenter to cuda:0
Hyperparameters read from the model weights: C^k=64, C^v=512, C^h=64
Single object mode: False
Traceback (most recent call last):
File "C:\Users\Holly\Documents\Track-Anything\app.py", line 372, in
model = TrackingAnything(SAM_checkpoint, xmem_checkpoint, e2fgvi_checkpoint,args)
File "C:\Users\Holly\Documents\Track-Anything\track_anything.py", line 20, in init
self.baseinpainter = BaseInpainter(self.e2fgvi_checkpoint, args.device)
File "C:\Users\Holly\Documents\Track-Anything\inpainter\base_inpainter.py", line 20, in init
net = importlib.import_module('inpainter.model.e2fgvi_hq')
File "C:\Users\Holly\AppData\Local\Programs\Python\Python310\lib\importlib_init_.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in load_unlocked
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "C:\Users\Holly\Documents\Track-Anything\inpainter\model\e2fgvi_hq.py", line 9, in
from inpainter.model.modules.feat_prop import BidirectionalPropagation, SecondOrderDeformableAlignment
File "C:\Users\Holly\Documents\Track-Anything\inpainter\model\modules\feat_prop.py", line 7, in
from mmcv.ops import ModulatedDeformConv2d, modulated_deform_conv2d
File "C:\Users\Holly\AppData\Local\Programs\Python\Python310\lib\site-packages\mmcv\ops_init.py", line 3, in
from .active_rotated_filter import active_rotated_filter
File "C:\Users\Holly\AppData\Local\Programs\Python\Python310\lib\site-packages\mmcv\ops\active_rotated_filter.py", line 10, in
ext_module = ext_loader.load_ext(
File "C:\Users\Holly\AppData\Local\Programs\Python\Python310\lib\site-packages\mmcv\utils\ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "C:\Users\Holly\AppData\Local\Programs\Python\Python310\lib\importlib_init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: DLL load failed while importing _ext: The specified procedure could not be found.
I configured the environment and successfully lauched in powershell , but the web interface cannot be opened under win11. Is it possible to make compatibility adjustments for Win11 in the future?
(venv) D:\Track-Anything-master>python app.py
Initializing BaseSegmenter to cuda:0
Traceback (most recent call last):
File "D:\Track-Anything-master\app.py", line 383, in
model = TrackingAnything(SAM_checkpoint, xmem_checkpoint, e2fgvi_checkpoint,args)
File "D:\Track-Anything-master\track_anything.py", line 18, in init
self.samcontroler = SamControler(self.sam_checkpoint, args.sam_model_type, args.device)
File "D:\Track-Anything-master\tools\interact_tools.py", line 37, in init
self.sam_controler = BaseSegmenter(SAM_checkpoint, model_type, device)
File "D:\Track-Anything-master\tools\base_segmenter.py", line 25, in init
self.model = sam_model_registrymodel_type
File "C:\ana\lib\site-packages\segment_anything\build_sam.py", line 15, in build_sam_vit_h
return _build_sam(
File "C:\ana\lib\site-packages\segment_anything\build_sam.py", line 105, in _build_sam
state_dict = torch.load(f)
File "C:\ana\lib\site-packages\torch\serialization.py", line 777, in load
with _open_zipfile_reader(opened_file) as opened_zipfile:
File "C:\ana\lib\site-packages\torch\serialization.py", line 282, in init
super(_open_zipfile_reader, self).init(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
是torch版本不对吗?
Hi,
Nice work!
Also, I often get the following error
in generate_video_from_frames
frames = torch.from_numpy(np.asarray(frames))
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (200,) + inhomogeneous part.
the console givesme this messages:
Initializing BaseSegmenter to cuda:0
Hyperparameters read from the model weights: C^k=64, C^v=512, C^h=64
Single object mode: False
Traceback (most recent call last):
File "G:\TrackAnything\Track-Anything\app.py", line 384, in
model = TrackingAnything(SAM_checkpoint, xmem_checkpoint, e2fgvi_checkpoint,args)
File "G:\TrackAnything\Track-Anything\track_anything.py", line 20, in init
self.baseinpainter = BaseInpainter(self.e2fgvi_checkpoint, args.device)
File "G:\TrackAnything\Track-Anything\inpainter\base_inpainter.py", line 20, in init
net = importlib.import_module('inpainter.model.e2fgvi_hq')
File "C:\Users\Tanvir\AppData\Local\Programs\Python\Python310\lib\importlib_init_.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in load_unlocked
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "G:\TrackAnything\Track-Anything\inpainter\model\e2fgvi_hq.py", line 9, in
from inpainter.model.modules.feat_prop import BidirectionalPropagation, SecondOrderDeformableAlignment
File "G:\TrackAnything\Track-Anything\inpainter\model\modules\feat_prop.py", line 7, in
from mmcv.ops import ModulatedDeformConv2d, modulated_deform_conv2d
File "C:\Users\Tanvir\AppData\Local\Programs\Python\Python310\lib\site-packages\mmcv\ops_init.py", line 2, in
from .active_rotated_filter import active_rotated_filter
File "C:\Users\Tanvir\AppData\Local\Programs\Python\Python310\lib\site-packages\mmcv\ops\active_rotated_filter.py", line 10, in
ext_module = ext_loader.load_ext(
File "C:\Users\Tanvir\AppData\Local\Programs\Python\Python310\lib\site-packages\mmcv\utils\ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "C:\Users\Tanvir\AppData\Local\Programs\Python\Python310\lib\importlib_init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: DLL load failed while importing _ext: The specified procedure could not be found.
Hello,
Your segmentation and tracking are impressive! I was wondering if there is a way to remove the background once a 'subject' is being tracked?
Thank you.
The app loads the videos correctly and allows me to select masks for tracking in video. However, when running the tracking feature, I keep getting this error:
File "app.py", line 351, in generate_video_from_frames
torchvision.io.write_video(output_path, frames, fps=fps, video_codec="libx264")
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\torchvision\io\video.py", line 70, in write_video
stream = container.add_stream(video_codec, rate=fps)
File "av\container\output.pyx", line 101, in av.container.output.OutputContainer.add_stream
File "av\utils.pyx", line 57, in av.utils.to_avrational
OverflowError: Python int too large to convert to C long
I uploaded a video, but it won't show.Get the info error.
Hello, thanks for the library.
Could you please provide version of Python and requirements.txt
with fixed versions?
Getting errors installing the package as it is.
Even better if you could provide a docker container.
(base) root@185:~/Track-Anything# python app.py --device cuda:0 --sam_model_type vit_h --port 80
Traceback (most recent call last):
File "/root/Track-Anything/app.py", line 1, in
import gradio as gr
File "/root/anaconda3/lib/python3.10/site-packages/gradio/init.py", line 3, in
import gradio.components as components
File "/root/anaconda3/lib/python3.10/site-packages/gradio/components.py", line 34, in
from gradio_client import utils as client_utils
File "/root/anaconda3/lib/python3.10/site-packages/gradio_client/init.py", line 1, in
from gradio_client.client import Client
File "/root/anaconda3/lib/python3.10/site-packages/gradio_client/client.py", line 21, in
from huggingface_hub import SpaceStage
ImportError: cannot import name 'SpaceStage' from 'huggingface_hub' (unknown location)
installing reqs ends with this:
active_rotated_filter.cpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\include\yvals.h(17): fatal error C1083: Cannot open include file: 'crtdbg.h': No such file or directory
error: command 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\bin\HostX86\x64\cl.exe' failed with exit code 2
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for mmcv-full
Running setup.py clean for mmcv-full
Successfully built metaseg thinplate segment-anything
Failed to build pycocotools mmcv-full
ERROR: Could not build wheels for pycocotools, mmcv-full, which is required to install pyproject.toml-based projects
C:\Users\XXXXX\Documents\Github\track-anything>
Why is this using requirements.txt?
Please move to poetry.
Added video 10 second 720p video, selected 3 masks, clicked Tracking, success:
Clicked inpaint, error like Error! You are trying to inpaint without masks input. Please track the selected mask first, and then press inpaint. If VRAM exceeded, please use the resize ratio to scaling down the image size.
Is this the correct order of operations? Is there a step I'm missing to send the same masks that just worked for tracking to inpaint?
I think I am the first person to report this issue.
template_frame.select( fn=sam_refine, inputs=[video_state, point_prompt, click_state, interactive_state], outputs=[template_frame, video_state, interactive_state, run_status] )
Regarding this piece of code, I encountered the following problem:
AttributeError: 'Image' object has no attribute 'select'
I have tried updating the Gradio version, but it didn't work. After removing the template_frame.select function, the code runs successfully, but it obviously loses an important feature. Is there a good solution to this problem?
For reference, my project is using the AutoDL platform.
Thanks for your work. I was wondering if there is any support for automatic multiple object tracking?
I really liked your work 💯 I want to add it to the Hugging Face platform. In this space there is the A10 Large GPU.
Hello,
I cloned the repo on my local machine but I get an ERROR on the GUI whenever I click TRACK
[ERROR:[email protected]] global D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap.cpp (166) cv::VideoCapture::open VIDEOIO(CV_IMAGES): raised OpenCV exception:
OpenCV(4.6.0) D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_images.cpp:293: error: (-215:Assertion failed) !_filename.empty() in function 'cv::CvCapture_Images::open'
Traceback (most recent call last):
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\gradio\routes.py", line 395, in run_predict output = await app.get_blocks().process_api(
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\gradio\blocks.py", line 1193, in process_api
result = await self.call_function(
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\gradio\blocks.py", line 916, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\anyio\to_thread.py", line 28, in run_sync
return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\anyio_backends_asyncio.py", line 818, in run_sync_in_worker_thread
return await future
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\anyio_backends_asyncio.py", line 754, in run
result = context.run(func, *args)
File "D:\Track-Anything\app.py", line 88, in get_frames_from_video
image_size = (frames[0].shape[0],frames[0].shape[1])
IndexError: list index out of range
Traceback (most recent call last):
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\gradio\routes.py", line 395, in run_predict output = await app.get_blocks().process_api(
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\gradio\blocks.py", line 1193, in process_api
result = await self.call_function(
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\gradio\blocks.py", line 916, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\anyio\to_thread.py", line 28, in run_sync
return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\anyio_backends_asyncio.py", line 818, in run_sync_in_worker_thread
return await future
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\anyio_backends_asyncio.py", line 754, in run
result = context.run(func, *args)
File "D:\Track-Anything\app.py", line 221, in vos_tracking_video
masks, logits, painted_images = model.generator(images=following_frames, template_mask=template_mask)
File "D:\Track-Anything\track_anything.py", line 44, in generator
mask, logit, painted_image = self.xmem.track(images[i], template_mask)
File "C:\Users\Francesco\anaconda3\envs\stable_diffusion\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "D:\Track-Anything\tracker\base_tracker.py", line 82, in track
probs, _ = self.tracker.step(frame_tensor, mask, labels) # logits 2 (bg fg) H W
File "D:\Track-Anything/tracker\inference\inference_core.py", line 102, in step
value, hidden = self.network.encode_value(image, f16, self.memory.get_hidden(),
File "D:\Track-Anything/tracker\model\network.py", line 75, in encode_value
others = torch.cat([
RuntimeError: torch.cat(): expected a non-empty list of Tensors
First of all, you guys did an amazing job 👏💯💯💯👏!!
I've always wanted to track objects in videos found other projects which can do pretty good job for tracking, but not able to run on Arm based edge devices e.g. RPi.
So just wondering is TAM for tracking purpose able to be run on Raspberry Pi 4(Mine has 4GB memory)?
Looking forward to your reply.
Cheers,
Winston
Hey guys,
is the inpainter(https://github.com/gaomingqi/Track-Anything/tree/master/inpainter) origin from this repo or from other project (standalone)?
thanks.
Can this project handle videos composed of image sequences? Does the segmentation map output by SAM contain category information?
However, the paper presents the quantitative results of the model on DAVIS. What can I do to test other video datasets, such as YouTube-VOS or YouTube-VIS, to obtain performance metrics on these datasets?
Traceback (most recent call last):
File "H:\Deepfacelab\Deepface\Track-Anything\venv\Lib\site-packages\gradio\routes.py", line 395, in run_predict
output = await app.get_blocks().process_api(
File "H:\Deepfacelab\Deepface\Track-Anything\venv\Lib\site-packages\gradio\blocks.py", line 1193, in process_api
result = await self.call_function(
File "H:\Deepfacelab\Deepface\Track-Anything\venv\Lib\site-packages\gradio\blocks.py", line 916, in call_function
prediction = await anyio.to_thread.run_sync(
File "H:\Deepfacelab\Deepface\Track-Anything\venv\Lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "H:\Deepfacelab\Deepface\Track-Anything\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "H:\Deepfacelab\Deepface\Track-Anything\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
result = context.run(func, *args)
File "H:\Deepfacelab\Deepface\Track-Anything\app.py", line 149, in vos_tracking_video
masks, logits, painted_images = model.generator(images=following_frames, template_mask=template_mask)
File "H:\Deepfacelab\Deepface\Track-Anything\track_anything.py", line 46, in generator
mask, logit, painted_image = self.xmem.track(images[i], template_mask)
File "H:\Deepfacelab\Deepface\Track-Anything\venv\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "H:\Deepfacelab\Deepface\Track-Anything\tracker\base_tracker.py", line 81, in track
probs, _ = self.tracker.step(frame_tensor, mask, labels) # logits 2 (bg fg) H W
File "H:\Deepfacelab\Deepface\Track-Anything/tracker\inference\inference_core.py", line 113, in step
return unpad(pred_prob_with_bg, self.pad), None
File "H:\Deepfacelab\Deepface\Track-Anything/tracker\util\tensor_util.py", line 35, in unpad
if len(img.shape) == 4:
AttributeError: 'NoneType' object has no attribute 'shape'
Gradio demo seems too slow. Any plans to have a Google colab demo?
When I clicked on single point, the SAM output like this:
it generates many masks on person
and then run tracker, got error:
tracker\util\mask_mapper.py:47 in convert_mask │
│ │
│ 44 │ │ │
│ 45 │ │ new_labels = list(set(labels) - set(self.labels)) │
│ 46 │ │ if not exhaustive: │
│ ❱ 47 │ │ │ assert len(new_labels) == len(labels), 'Old labels found in non-exhaustive m │
│ 48 │ │ │
│ 49 │ │ # add new remappings │
│ 50 │ │ for i, l in enumerate(new_labels): │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AssertionError: Old labels found in non-exhaustive mode
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.