kevin-thu / diffmorpher Goto Github PK

View Code? Open in Web Editor NEW

336.0 336.0 35.0 44.93 MB

Official Code for DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing (CVPR 2024)

License: Other

Python 100.00%

diffmorpher's People

Contributors

Stargazers

Watchers

diffmorpher's Issues

Request to add support for API？

Request to add support for API. I have a website that specializes in introducing AI-related applications. I hope to be able to add API call support. Thank you, brother.

Just wanted to know if there is a custom node for this, This is somewhat very alluring technique. I always wanted a image to image interpolation workflow and custom node. Using multiple key frames we can animate short stories and illustrations. This is really amazing

errors while running code on gradio

Hello.
I've followed pretty simple installation rules and I get following errors while executing DiffMorpher.
The first time I've cloned the repo I did not download pretrained models from hugginface and on the first run it did it by itself. When I hit Run it succesfully created LoRas, but on the execution it failed.
After that I've downloaded pretrained models repo and placed it inside DiffMorpher/stabilityai/stable-diffusion-2-1-base folder.

This is what I get on miniconda prompt

(mrp) F:\ComfyUI\DiffMorpher>python app.py
Setting up [LPIPS] perceptual loss: trunk [alex], v[0.1], spatial [off]
C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=AlexNet_Weights.IMAGENET1K_V1`. You can also use `weights=AlexNet_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Loading model from: C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\lpips\weights\v0.1\alex.pth
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Traceback (most recent call last):
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\queueing.py", line 456, in call_prediction
    output = await route_utils.call_process_api(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\route_utils.py", line 232, in call_process_api
    output = await app.get_blocks().process_api(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1522, in process_api
    result = await self.call_function(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1144, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 2134, in run_sync_in_worker_thread
    return await future
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py", line 674, in wrapper
    response = f(*args, **kwargs)
  File "F:\ComfyUI\DiffMorpher\app.py", line 47, in run_diffmorpher
    morpher_pipeline = DiffMorpherPipeline.from_pretrained(model_path, torch_dtype=torch.float32).to("cuda")
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1101, in from_pretrained
    model = pipeline_class(**init_kwargs)
  File "F:\ComfyUI\DiffMorpher\model.py", line 108, in __init__
    super().__init__(vae, text_encoder, tokenizer, unet, scheduler,
TypeError: StableDiffusionPipeline.__init__() takes from 8 to 9 positional arguments but 10 were given
Traceback (most recent call last):
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\queueing.py", line 456, in call_prediction
    output = await route_utils.call_process_api(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\route_utils.py", line 232, in call_process_api
    output = await app.get_blocks().process_api(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1522, in process_api
    result = await self.call_function(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1144, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 2134, in run_sync_in_worker_thread
    return await future
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py", line 674, in wrapper
    response = f(*args, **kwargs)
  File "F:\ComfyUI\DiffMorpher\app.py", line 47, in run_diffmorpher
    morpher_pipeline = DiffMorpherPipeline.from_pretrained(model_path, torch_dtype=torch.float32).to("cuda")
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1101, in from_pretrained
    model = pipeline_class(**init_kwargs)
  File "F:\ComfyUI\DiffMorpher\model.py", line 108, in __init__
    super().__init__(vae, text_encoder, tokenizer, unet, scheduler,
TypeError: StableDiffusionPipeline.__init__() takes from 8 to 9 positional arguments but 10 were given

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\queueing.py", line 501, in process_events
    response = await self.call_prediction(awake_events, batch)
  File "C:\Users\w1xmo\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\queueing.py", line 465, in call_prediction
    raise Exception(str(error) if show_error else None) from error
Exception: None

Is there an approximate date of code releasing?

Hi, there. Great Work! Remarkable effect seen in your demo. Wondering is there an approximate date of code releasing?

Inference code and weights

Hi Authors, This is excellent work. WOuld you be releasing weights and inference code anytime soon?

dose it support Mac M1 ?

extrapolate images

incredible work, I was wondering if it is possible to go further and extrapolate images in both directions

Other diffusion model support

Hi, there. Will you support other diffusion models like XL soon?

using different stable diffusion models

Can I use the stable diffusion models from civit.ai instead of the stabilityai/stable-diffusion-2-1-base model in my local project?
How can I solve this problem?
Thanks

repeat model train

I want to use this method for more than one image of the same person. Do I need to do lora training every time? When I try it, success decreases in images without lora training. Does training lora with multiple images solve this problem? If so, how can I do this?
Thanks

click train loraA occurred an error

platform: windows 10
conda python: 3.10.13
pip: pip 23.1.2
Nvidia: 3060 12G

Wrong color result

I input 2 images with black background and white object(with light and shading). The result shows wrong color.

input

output

I have tried convert the encoding of images( RGB, BGR, uint8, uint16) and the format(.png, .jpg). But those did not work.
So what do you think may be the reason?
Thanks

Additional size output and CRF values.

Love the work and would appreciate being able to set larger/different output file sizes up to 1920 x 1088. Also like to have option to change compression rates( CRF) - my last output was 3,000 and need at least 50k. Would also be very excellent if one could morph between a larger set of input files - 4 - 6 maybe? Thanks for your efforts.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.