Comments (6)
It should, you're right.
Are you running with variable WORKSPACE_SYNC=true for the serverless worker?
I suspect the problem is that the worker is trying to use the on-image python environment rather than the synced workspace version. That shouldn't happen so I'll look into it asap.
I'm not a fan of the workspace sync though because RunPod disks are slow and it adds more runtime completely Vs building the image with required nodes and models.
from comfyui.
Hi. Thanks for your reply.
I didn't try to turn WORKSPACE_SYNC=true on the serverless worker yet. Did that and it works!
I set it all up mostly following this tutorial and was assuming that WORKSPACE_SYNC has to be always off on the serverless template: https://medium.com/@iknowkungfu/setup-complex-ai-image-workflows-at-scale-using-a-fleet-of-cloud-gpus-and-comfyui-1ac9a6c4b0b3
It worked with setting up on a regular pod with sync=on for the initial setup and turning off for serverless but broke on updating the backend.
I agree that ideally I would build images including everything but since I run quite complicated workflows with many custom nodes this largely exceeds my Docker capabilities.
Thanks a lot for your great work!
from comfyui.
However, I am gettin a lot of 'Server not ready after timeout (30s)' errors now.
Probably will have to look into building a custom Docker image after all.
from comfyui.
The timeout is coming from the current mechanism for switching the environment from /opt to /workspace - It's a bad design decision that I plan to work around.
Currently the /opt/micromamba directory is removed and replaced with a symlink (to /workspace/environments/...), but this can take some time on slow disks. The better solution is to have the environment always set by symlinks as there will not be a delete step.
My fault but I'll fix it as soon as I can - it's still better to build the image though because networked disks aren't great for this usecase.
Really, syncing part of image so that it can be modified throws away much of the benefits that docker brings. I understand why users want to do it, I just think it introduces more points of failure which I'd never want in a production application. I will add warnings to the documentation
from comfyui.
Thanks.
I started building a custom Docker image based on ai-dock.
I made a little script, which analyzes Comfy workflows and lists the custom node Githubs.
Added them to .../layer1/init.sh
NODES=(
"https://github.com/ltdrdata/ComfyUI-Manager"
"https://github.com/Extraltodeus/ComfyUI-AutomaticCFG",
"https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved",
"https://github.com/Seedsa/Fooocus_Nodes",
"https://github.com/chrisgoringe/cg-use-everywhere",
)
However, the custom nodes seem to not be available to the worker.
Do I need to remove WORKSPACE=/runpod-volume from the environment variables? Or still copy the nodes somewhere into the docker image?
from comfyui.
No longer supporting old builds due to project direction changes. New API wrapper will be documented soon for all serverless use cases
from comfyui.
Related Issues (20)
- Custom nodes, imports fails. Moving mamba env to workspace? HOT 14
- Server not ready after timeout ({timeout}s HOT 7
- mamba-skel not found HOT 1
- Progress updates from RunPod serverless while generating HOT 3
- docker compose build error - run set -eo pipefall && ... HOT 4
- Getting error "value_not_in_list", i.e. checkpoint is not getting loaded somehow. HOT 6
- [HELP with __init__.py] I have the problem with the following Custom Nodes, which when I want to install manually, I get an error HOT 1
- endpoint.run not working HOT 3
- How do I get this working with an Intel GPU? HOT 4
- How to Setup serverless with custom nodes and models HOT 2
- PermissionError some custom nodes HOT 2
- Taking too long to connect to ComfyUI on Runpod HOT 1
- What is the correct path for the document HOT 1
- How Can I link with Krita AI Plugin HOT 2
- Newbie question: Run a serverless on-demand API backend on runpod.io or vast.ai HOT 2
- New release won't initialise on RunPod HOT 3
- DeepFuze node on comfy ui. HOT 2
- Newbie question: light customization using ai-dock/comfyui as base
- Bypass ai-dock authentication for api requests to Comfy (runpod) HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from comfyui.