Comments (7)
Converting through dynamo is a royal PITA (remove/comment out sdpa context managers, replace torch.nn.functional.scaled_dot_product_attention with the equivalent python code from the docs), but the "legacy" onnx exporter seems to work fairly well.
The graph isn't the prettiest and I haven't checked the output for correctness, but this should do the trick:
import torch
from Models import VisionModel
model = VisionModel.load_model("../joytag_weights/")
torch_input = torch.randn(1, 3, 448, 448)
torch.onnx.export(
model,
({"image": torch_input}, {}),
"joytag.onnx",
export_params=True,
do_constant_folding=True,
input_names=["input"],
output_names=["output"],
dynamic_axes={"input": {0: "batch_size"}, "output": {0: "batch_size"}},
)
from joytag.
this is as far as I got so far...
python -m optimum.exporters.onnx --model joytag/ onnx/ --library transformers --task image-classification --framework tf
I had to add "model_type": "vit" to the config.json
now it complains about missing metadata. Perhaps I am approaching it wrong, I am still learning about all this stuff.
Loading PyTorch model in TensorFlow before exporting.
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.11/site-packages/optimum/exporters/tasks.py", line 1822, in get_model_from_task
model = model_class.from_pretrained(model_name_or_path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.11/site-packages/transformers/modeling_tf_utils.py", line 2879, in from_pretrained
raise OSError(
OSError: The safetensors archive passed at joytag/model.safetensors does not contain the valid metadata. Make sure you save your model with the save_pretrained
method.
from joytag.
I tried to export an ONNX the other day, but it looks like PyTorch's ONNX support is still very alpha at the moment. I'll try again if I get time, but no guarantees for now.
from joytag.
Converting through dynamo is a royal PITA (remove/comment out sdpa context managers, replace torch.nn.functional.scaled_dot_product_attention with the equivalent python code from the docs), but the "legacy" onnx exporter seems to work fairly well.
The graph isn't the prettiest and I haven't checked the output for correctness, but this should do the trick:
import torch from Models import VisionModel model = VisionModel.load_model("../joytag_weights/") torch_input = torch.randn(1, 3, 448, 448) torch.onnx.export( model, ({"image": torch_input}, {}), "joytag.onnx", export_params=True, do_constant_folding=True, input_names=["input"], output_names=["output"], dynamic_axes={"input": {0: "batch_size"}, "output": {0: "batch_size"}}, )
Can you please be a bit more specific about how to do this? I can't figure out how to do it
from joytag.
Ok, the problem was Torch version. I had 2.0.0 in my machine since I can only run DirectML stuff, upgrading to Torch 2.2.0 fixed the problem and the code executed successfully.
from joytag.
https://huggingface.co/fancyfeast/joytag/blob/main/model.onnx
Thank you, SmilingWolf, that code worked on PyTorch 2.2.0. I just double checked the ONNX model's outputs to make sure it was working correctly.
I'll get some example usage code up and then close this issue.
from joytag.
I have a working implementation of JoyTag up and running in my tools with C# and the OnnxRuntime.
There is a small variation when compared to the Python implementation with the Safetensors model; but that seems to be the case for any model. The Onnx runtime usually get 1 or 2 extra tags (and its not even false positives) when running inference with the same threshold.
from joytag.
Related Issues (17)
- [Discussion] Comparison with Danbooru interrogator in SD Automatic1111 HOT 3
- Questions about choice of tags HOT 1
- Versions of the dependencies? HOT 1
- On validation metrics and thresholds HOT 2
- Training Support
- Great work. Will you release finetune code? HOT 3
- URL for download is broken
- where can I find top_tags.txt ? HOT 1
- This is a great project, and I am concerned about when it can be updated,
- `torch.backends.cuda.sdp_kernel()` is deprecated. use `torch.nn.attention.sdpa_kernel()`
- Great project! I created a .py script that tags all files in an \input\ folder HOT 4
- What is tag-machine? HOT 1
- it can be further improved
- "from Models import VisionModel" gives unresolved reference HOT 1
- Joytag Colab Notebook
- Discord server
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from joytag.