Comments (6)
I checked this earlier and got to work. As far as I can tell...300%! I could be wrong but as far as I can tell, we can now run ComfyUI natively!!!
But I'm not much of a technical person. But MLX can replace TorchSDE as far as I can tell. That is at least the last requirement of ComfyUI's base that wasn't native.
TorchSDE is solely utilized here https://github.com/comfyanonymous/ComfyUI/blob/248d9125b0821851ea4b7c749df20a040f5ebe57/comfy/k_diffusion/sampling.py#L6
I'm currently trying to patch things in but it is a mess. Maybe I'll be able to do it with the Suite's help. Was running into some issues because I was trying to mesh ComfyUI with coreml-stable-diff.
Edit: eh not so certain I'll be able to get it all together. But I imagine someone can. The attention.py module was tripping up my run. Need to mesh CoreML's with Comfyui's.
Edit 2: As far as the common extended features, I can say...
I know there's an onnx-runtime-silicon. I was using Miniforge so I just ran "onnx-runtime" and was able to get it. There's also onnx-coreml (in-lieu of onnx). Torchaudio is avail via Nightly conda install torchaudio -c pytorch-nightly
from comfyui-coremlsuite.
@BuildBackBuehler Thank you for your efforts! Have you been able to get any improvements on inference? Really excited to know :)
Like I said, I didn't get close to makin it work. However, I've been picking up a bit to understand the basics. It seems there is a way to simplify and semi-automate the process of converting all the Torch refs to MLX refs. Someone actually in the ML field would probably be able to do this with ease. I was working with LLMs and it seems MLX, Pytorch, Tensorflow all have a Common Dictionary that allows one to convert framework-to-framework. Just a total shot in the dark, but I presume after the conversion, it'd likely log all the inconsistencies/unsupported definitions that the target dictionary doesn't have one for.
But the Apple team posted some naked SD performance #s here – and as you can see, no Pytorch necessary!
from comfyui-coremlsuite.
@BuildBackBuehler Thank you for your efforts! Have you been able to get any improvements on inference? Really excited to know :)
from comfyui-coremlsuite.
Never mind my previous comment I forgot how god-awful SDXL is on Mac I've been using 1.5 forgot how slow it is ya MLX is a step in the right direction lol
from comfyui-coremlsuite.
Few notes, there seems to be a lack of safetensor support in MLX currently their working on adding support, theres actually also no support for coreml that i can tell...
I still think it would be possible to do but it's a lot of work i think as we couldn't rely on the base model references, might likely need to be a seperate undertaking like i said since i don't seem to think it works with coreml
from comfyui-coremlsuite.
anyone knows if there's a online community of mlx enthusiasts looking to integrate mlx into different AI applications like Automatic1111, comfyUI, Foooocus, ollama etc
from comfyui-coremlsuite.
Related Issues (20)
- Lora strengths should probably be in the names or somewhere HOT 2
- Long conditioning prompts cause crashes from tensor lengths not matching? HOT 3
- Facedetailer using the experimental adapter HOT 3
- SDXL Turbo support HOT 7
- I got this error after installing : ImportError: cannot import name 'ModelSamplingDiscreteLCM' from 'comfy_extras.nodes_model_advanced' HOT 7
- Question : what should i put in controlnet model .fp16.safetensors or .safetensors ? HOT 4
- Question: Is it feasible to implement AnimateDiff in CoreML? HOT 2
- Update of comfyui breaks the code HOT 1
- ANE not activated HOT 2
- EnumeratedShapes and Ranges... wow
- Loras causes model to produce garbage HOT 5
- Module will not load HOT 3
- Latent upscale has some problem
- Convert checkpoint: file not found HOT 4
- IPAdapter
- Chip M3max HOT 3
- Error when trying to sample with converted SD1.5 inpainting models: The size of tensor a (9) must match the size of tensor b (4) at non-singleton dimension 1 HOT 1
- How to Debug Core ML Adapter (Experimental) ? HOT 1
- Unable to generate proper images by the combined of LoRA and SD model on Mac HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from comfyui-coremlsuite.