xx025 / stable-video-diffusion-webui Goto Github PK

stable-video-diffusion-webui, img to videos| 图片生成视频

Python 99.30% CSS 0.70%

stable-video-diffusion-webui's Introduction

Stable-Video-Diffusion-WebUI


img	videos

Preparation

git clone https://github.com/xx025/stable-video-diffusion-webui.git
cd stable-video-diffusion-webui

Before starting, you need to create a virtual environment. It is recommended to use conda. If you are not comfortable using conda, create a Python 3.10 environment in your own way.

conda create -n svd python=3.10 
conda activate svd

Then execute the following commands in order,

# During this process, the torch version corresponding to cu118 will be installed. You can try installing it first. The author tested it on a machine with cuda11.6, and it installed normally. 
# If it doesn't fit your computer, please change the --extra-index-url in requirements.txt accordingly.
python install.py

# Create a folder to save weight files
mkdir checkpoints 
# Download the model. This may take some time.
# If it's inconvenient to use wget, you can download it manually and save it to the checkpoints folder with the name svd_xt.safetensors
wget https://huggingface.co/vdo/stable-video-diffusion-img2vid-xt/resolve/main/svd_xt.safetensors?download=true -P checkpoints/ -O svd_xt.safetensors

Run

During the first run, other model files will be downloaded.

python run.py

More setting

Please check settings.py and make modifications.

Communicate Group

Only for China users

Note

This library is modified from url, which is the colab version of stable-video-diffusion.

stable-video-diffusion-webui's People

Contributors

Stargazers

Watchers

stable-video-diffusion-webui's Issues

How to change the frame per second (FPS)？

The result video seems to be rendered at 6 fps always

OS system issue

Does this webui support Mac OS system with M series processor？

Output video has a weird pink filter

000001.mp4

Not sure what's causing this, but it would be greatly appreciated if any advice could be given as to how to fix this.

请问`python run.py`时下载的模型来自哪里？

你好，感谢你的开源精神，让我能早日尝鲜SVD👍

我的情况是这样的：我有一台服务器，能上网，但不方便科学上网；还有一台笔记本，能上网，也方便科学上网。
我在服务器里部署你的WebUI时，执行最后一步python run.py时，报错：

Traceback (most recent call last):
  File "/home/zyl/stable-video-diffusion-webui/run.py", line 6, in <module>
    from modules.model import infer
  File "/home/zyl/stable-video-diffusion-webui/modules/model.py", line 19, in <module>
    from modules.model_setting import device, model, filter_x
  File "/home/zyl/stable-video-diffusion-webui/modules/model_setting.py", line 22, in <module>
    model, filter_x = load_model(
  File "/home/zyl/stable-video-diffusion-webui/modules/utils.py", line 17, in load_model
    model = instantiate_from_config(config.model).to(device).eval().requires_grad_(False)
  File "/home/zyl/stable-video-diffusion-webui/generative-models/sgm/util.py", line 175, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/home/zyl/stable-video-diffusion-webui/generative-models/sgm/models/diffusion.py", line 59, in __init__
    self.conditioner = instantiate_from_config(
  File "/home/zyl/stable-video-diffusion-webui/generative-models/sgm/util.py", line 175, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/home/zyl/stable-video-diffusion-webui/generative-models/sgm/modules/encoders/modules.py", line 79, in __init__
    embedder = instantiate_from_config(embconfig)
  File "/home/zyl/stable-video-diffusion-webui/generative-models/sgm/util.py", line 175, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/home/zyl/stable-video-diffusion-webui/generative-models/sgm/modules/encoders/modules.py", line 1039, in __init__
    self.open_clip = instantiate_from_config(open_clip_embedding_config)
  File "/home/zyl/stable-video-diffusion-webui/generative-models/sgm/util.py", line 175, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/home/zyl/stable-video-diffusion-webui/generative-models/sgm/modules/encoders/modules.py", line 591, in __init__
    model, _, _ = open_clip.create_model_and_transforms(
  File "/home/zyl/miniconda3/envs/svd/lib/python3.10/site-packages/open_clip/factory.py", line 382, in create_model_and_transforms
    model = create_model(
  File "/home/zyl/miniconda3/envs/svd/lib/python3.10/site-packages/open_clip/factory.py", line 281, in create_model
    checkpoint_path = download_pretrained(pretrained_cfg, cache_dir=cache_dir)
  File "/home/zyl/miniconda3/envs/svd/lib/python3.10/site-packages/open_clip/pretrained.py", line 552, in download_pretrained
    target = download_pretrained_from_hf(model_id, cache_dir=cache_dir)
  File "/home/zyl/miniconda3/envs/svd/lib/python3.10/site-packages/open_clip/pretrained.py", line 522, in download_pretrained_from_hf
    cached_file = hf_hub_download(model_id, filename, revision=revision, cache_dir=cache_dir)
  File "/home/zyl/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/zyl/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1377, in hf_hub_download
    raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

连不上huggingface，也就是你提到的需要科学上网，下载其他的模型。我能想到的围魏救赵方式是用笔记本把这些模型下载下来，然后上传到服务器里。所以问题就来了：

我需要下载哪些模型呢？

谢谢！

多nv卡的情况可以通过配置来提升生成的总帧数么

因为如果只是修改frame的参数会导致内存溢出。所以想请教下两张或多张nv卡的情况是否有办法可以提升显存大小

问题

每次跑跑就没内存了，怎么储存到数据盘而不是系统盘当中

Can the generated dynamics guided by text?

Hi @xx025 , I read the paper, and found that the video can be controlled by text. However, I didn't find how to realize this function in their repo and this repo. Can you give me some hints?

第一次运行run.py时无法连接huggingface.co下载open_clip

你好，我在第一次运行run.py时每次都会报错network is unreachable，下载不了open_clip。将setting.py中的

environ['http_proxy'] = 'http://127.0.0.1:7890/'

os.environ['https_proxy'] = 'http://127.0.0.1:7890/'

取消注释后会报错拒绝访问。
我现在直接在本地下载了open_clip_pytorch_model.bin这个文件，请问要放到哪个目录下面哇。
或者如何解决无法连接huggingface的问题
谢谢！

SVD vs AnimateDiff

Hello,

First of all, thank you for making this. Espacially since it's made in Gradio.

A lot of people want to mess around with SVD and this is simpler to work with.

I'd like to ask you if you compared generation results with AnimateDiff (and Deforum) for quality or other aspects.

Thanks,
Good day

使用CG客户端下载镜像到本地运行，默认运行时吃满了CPU，GPU基本没变化

请问该如何切换到GPU运行呢，或者我该提供些什么更多的信息。