chongzhou96 / edgesam Goto Github PK

https://github.com/slz929/EdgeSAM-MNN

As mentioned in the title and how?

Hi, @chongzhou96, thanks a lot for your outstanding work!

I've successfully integrated EdgeSAM into X-AnyLabeling, and your open-source contribution has been invaluable for achieving efficient execution on devices with limited computational resources.

Hello! Thanks a lot for your great work! We've already supported grounded-edga-sam demo in Grounded-Segment-Anything !

I'd like to finetune EdgeSAM for a specific task.
Is there code already available?
Have you experience in finetuning EdgeSAM?

感谢edgeSam提供了边缘端及移动端的sam方案，目前想使用tensorrt&rknn&altas进行部署和推理，请问有相应的参考资料么？

Hello! I'm interested in following your work. May I kindly inquire whether the code you use for training will be made open source? Thank you.

wget -P weights/ https://huggingface.co/spaces/chongzhou/EdgeSAM/resolve/main/weights/edge_sam_3x_encoder.onnx
这个地址链接不上，请问还可以在哪下载模型文件呀？

The performance demonstrated in the paper for the iPhone 14 is remarkable, and I'm currently attempting to evaluate the model on an entry-level Android phone.

In a basic implementation, the encoder_session.run() operation takes approximately 3,000-4,000 ms for 720x540 images on a Samsung A20 (using the CPU execution provider) as the official ONNX model files do not support NNAPI. NNAPI is the Neural Network API leveraging the GPU on Android.

I'm curious about the significant difference in encoder performance between the iPhone 14, achieving 70 FPS and 14 ms per image, and the Samsung A20, taking 3,000-4,000 ms per image.

Could you please provide some advice on how to address this performance gap?

[Environment]
My testing setup involves ONNX files from the official link: EdgeSAM Encoder and EdgeSAM Decoder.
The tests are conducted using ONNX Runtime version 1.16.3 on a Samsung A20 with Android 11 (CPU: Exynos 7884, GPU: Mali-G71 MP2).

I appreciate your assistance and thank you for your valuable contributions.

Hi thanks for your work, I would like to ask if the license will also apply for the model itself intended as weights values or is only applicable to code, because for me it is not clear from your license file for commercial purpose?

Thanks

Thanks for your great work.
Our C++ code corresponded to your Edge SAM models.

Here is the Segment Anything CPP Wrapper for macOS.

Please ask about your license.
We are going to implement the EdgeSAM models on RectLabel macOS apps.
Our total revenue is about $1K/month from RectLabel apps on Mac App Store.
Can we implement the EdgeSAM models on our apps?
Please let us know your feedback.

I don't want to manually set points or boxes, I want to directly segment all the objects in the graph

你好，工作非常出色！！
但是关于论文中提及的RPN Module似乎代码没有实现，权重文件也未给出， 是出于什么样的考虑呢～ 期待回复

Does your model not support torch.jit? I encounter an error when trying to convert the model with the following code. If there is a solution, please let me know.

import torch
from edge_sam import SamPredictor, sam_model_registry

sam_checkpoint = "./weights/edge_sam.pth"
model_type = "edge_sam"

# load model and run
sam = sam_model_registry[model_type](checkpoint=sam_checkpoint)
sam_jit = torch.jit.script(copy.deepcopy(sam))

Will you release it?

Hello EdgeSAM Team,

I hope this message finds you well. I am reaching out to request the integration of NCNN into the EdgeSAM project. NCNN, known for its high performance in neural network inference on mobile devices, seems like a perfect fit for EdgeSAM's objectives in edge computing scenarios.

The inclusion of NCNN would provide several benefits, including:

1. Enhanced Performance: Leveraging NCNN's optimized routines could improve the efficiency and speed of EdgeSAM's computations, especially on devices with limited resources.

2. Broader Compatibility: With NCNN's support for multiple platforms, EdgeSAM could extend its usability across a wider range of mobile and embedded devices.

3. Community Engagement: Integrating popular and widely-used frameworks like NCNN can attract more contributors and users to the EdgeSAM project, fostering a larger community and encouraging collaborative development.

I believe that the synergy between EdgeSAM and NCNN could lead to significant advancements in edge computing applications. I am looking forward to your thoughts on this suggestion and hope to see a collaboration that benefits both projects and their respective communities.

Thank you for considering my request.

Best regards,
Umit

When running the decoder with the same number of points time after time, the running time may approach the number reported by the performance analysis of Xcode. However, this is not the usual use case of this model.
Usually, one would select points iteratively, such that the size of the model input constantly changes (first 256x64x64, 1x1x2, 1x1, than 256x64x64, 1x2x2, 1x2 and so on).
Every time the model is used with a different size, some internal CoreML state is discarded, and the running time is that of a first-run (which is ~10x slower!).

If the model can be designed such that it is constantly run with the same number of points (16), with some of the points being ignored, perhaps it could help resolve this issue (but I really have no idea if that's possible).

Segmentation labeling tool ISAT has supported EdgeSAM.

Hello, how can I predict the whole picture without using a mouse click to determine the segmentation coordinate point?

could you provide the train code? thank you very mush