tencentarc / photomaker Goto Github PK

The main problem of how to process FaceID now relies merely on the difference of details strength between eyes and others.
Yet the purely mixture of texture inversion would mix all the 3d distortion together adding artifacts into your model.
Thus I suggest, maybe adding 3d relation to the training would help.

I did'nt find setup.py and command run error

Traceback (most recent call last):
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
raise err
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
conn.connect()
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\connection.py", line 309, in connect
conn = self._new_conn()
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\connection.py", line 164, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x0000018A193E0220>, 'Connection to huggingface.co timed out. (connect timeout=10)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\requests\adapters.py", line 440, in send
resp = conn.urlopen(
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
retries = retries.increment(
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\urllib3\util\retry.py", line 446, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /TencentARC/PhotoMaker/resolve/main/photomaker-v1.bin (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000018A193E0220>, 'Connection to huggingface.co timed out. (connect timeout=10)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\huggingface_hub\file_download.py", line 1238, in hf_hub_download
metadata = get_hf_file_metadata(
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\huggingface_hub\file_download.py", line 1631, in get_hf_file_metadata
r = _request_wrapper(
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\huggingface_hub\file_download.py", line 385, in _request_wrapper
response = _request_wrapper(
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\huggingface_hub\file_download.py", line 408, in _request_wrapper
response = get_session().request(method=method, url=url, **params)
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\requests\sessions.py", line 529, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\requests\sessions.py", line 645, in send
r = adapter.send(request, **kwargs)
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\huggingface_hub\utils_http.py", line 67, in send
return super().send(request, *args, **kwargs)
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\requests\adapters.py", line 507, in send
raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /TencentARC/PhotoMaker/resolve/main/photomaker-v1.bin (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000018A193E0220>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: d88ac3bf-fb95-4530-ae47-b0a18dc9d732)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "gradio_demo/app.py", line 24, in
photomaker_ckpt = hf_hub_download(repo_id="TencentARC/PhotoMaker", filename="photomaker-v1.bin", repo_type="model")
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "C:\Users\abc\anaconda3\envs\p\lib\site-packages\huggingface_hub\file_download.py", line 1371, in hf_hub_download
raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

ValueError: PhotoMaker currently does not support multiple trigger words in a single prompt. Trigger word: img, Prompt: a beautiful girl on Italy street, best quality.

this is my question but I installed this package

如此5张图

差距比较大啊

They do not work in resolutions other than 1024
try 768 and 512 and it just generates random noise

Hi! Awesome work! I want to train on other data, are you planning to release a training code?

Is there anyway I can use this with automatic 1111 webui?

Hello,

Thank you for sharing this exciting looking project.

I am having issues running this locally on my windows machine. Having reviewed other issues raised, I have got the gradio demo running, however, no matter what my outputs look like this:

Do you have any thoughts on what might be going wrong?

Thank you in advance.

We have seen very good results on SDXL. How does this method perform on SD1.5? Are there any plans to offer a version for SD1.5?

Thank you very much.

demo notebooks don't work at all, when will they be fixed ?!

Can I customize the model?

Thanks a lot for releasing code and models for such an amazing research!

I have been trying to run the code but experiencing following error:
'''
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/release_model/resolve/main/photomaker-v1.bin
'''

It looks like I fail download model from huggingface repo. My question is I have successfully download your model through
'''
photomaker_path = hf_hub_download(repo_id="TencentARC/PhotoMaker", filename="photomaker-v1.bin", repo_type="model")
'''
to 'HuggingfaceCache/hub', may I ask if I could use that model directly?

I am new to huggingface. I really anticipate your reply and experience your masterpiece.

amazing work! is it possible for you to share the prompts used for the images in the Stylization grid in the official project page?

run photomaker_demo code, inference with 2 input img, 40 steps, generete 4 img need 4 minutes on V100

RuntimeError: Failed to import diffusers.schedulers.scheduling_euler_discrete because of the following error (look up to see its traceback):
cannot inherit non-frozen dataclass from a frozen one

What is the license here, and can it be used for commercial purposes?

Awesome work!
I have read your paper and noticed that your training data is extracted from VGG2 and Voxceleb, and substantial efforts have been put into it. Could you please let me know if there are any plans to release the dataset?

As per title. If not, I'm new to diffusers, if anyone can guide me. Can it be used for let's say with Segmind SSD 1B and this used as an IP adapter in a gradio UI?

C:\AI\PhotoMaker\PhotoMaker>pip install -r requirements.txt
Requirement already satisfied: torch==2.0.1 in c:\users\henso\appdata\local\programs\python\python310\lib\site-packages (from -r requirements.txt (line 1)) (2.0.1)
Requirement already satisfied: torchvision==0.15.2 in c:\users\henso\appdata\local\programs\python\python310\lib\site-packages (from -r requirements.txt (line 2)) (0.15.2)
ERROR: Could not find a version that satisfies the requirement pytorch-cuda==11.8 (from versions: none)
ERROR: No matching distribution found for pytorch-cuda==11.8

C:\AI\PhotoMaker\PhotoMaker>

I tried editing the requirements file but still ran into issues. any solutions?

civitai_models/sdxlUnstableDiffusers_v11.safetensors
civitai_models/xl_more_art-full.safetensors
两个模型 hug上没有找到，能否提供相关下载信息？

I have little doubt the team behind PhotoMaker or another team is working on or will soon be working on this, but I wanted to ask anyway: Is there any plans to create a plugin for WebUI so this can slot seamlessly into Auto1111's WebUI and/or other such WebUIs?

Asking on behalf of the Umi AI team.

thank you for release photoMaker!

I tested photo maker demo with 5 peoples face image in the huggingface demo

but I got bad result than IP-adapter face id

Are other people getting good results when testing?

I am getting that a error. Can anyone help me?

(base) C:\Users\QWERTY\Desktop\PhotoMaker-main>python gradio_demo/app.py
Traceback (most recent call last):
File "C:\Users\QWERTY\Desktop\PhotoMaker-main\gradio_demo\app.py", line 10, in
import spaces
File "C:\Users\QWERTY\anaconda3\Lib\site-packages\spaces_init_.py", line 10, in
from .zero.decorator import GPU
File "C:\Users\QWERTY\anaconda3\Lib\site-packages\spaces\zero\decorator.py", line 18, in
from .wrappers import regular_function_wrapper
File "C:\Users\QWERTY\anaconda3\Lib\site-packages\spaces\zero\wrappers.py", line 42, in
Process = multiprocessing.get_context('fork').Process
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\QWERTY\anaconda3\Lib\multiprocessing\context.py", line 243, in get_context
return super().get_context(method)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\QWERTY\anaconda3\Lib\multiprocessing\context.py", line 193, in get_context
raise ValueError('cannot find context for %r' % method) from None
ValueError: cannot find context for 'fork'

Requirement already satisfied: torch==2.0.1 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from -r requirements.txt (line 1)) (2.0.1)
Requirement already satisfied: torchvision==0.15.2 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from -r requirements.txt (line 2)) (0.15.2)
ERROR: Could not find a version that satisfies the requirement pytorch-cuda==11.8 (from versions: none)
ERROR: No matching distribution found for pytorch-cuda==11.8

what is this problem?

Hi, thanks for your great work. In the paper, it is said in section 3.2, that word feature vector will be fused with image embeddings. Could you please provide more details about such fusion? Is it a concat operation on channel dim?

when I run pyton app.py,got this error.

For windows users out there, I have created a fork with an easy set of instructions and updates to facilitate the use on windows. You can find the 1st release at:

https://github.com/bmaltais/PhotoMaker

This fork also support non square output format so you can produre results like:

You will find the width and height setting under the Advanced Accordion in the GUI:

Once you have followed the installation instructions you simply start the Gradio GUI with: .\gui.bat

Hi, thanks for your great work! May I ask how many images did you use to train the model?

I've cloned PhotoMaker and installed a venv of Python 3.8.
All requirements has installed ok with help of an earlier version of pytorch since I got this first:

(venv) Q:\PhotoMaker>pip install -r requirements.txt
Collecting torch==2.0.1
Downloading torch-2.0.1-cp38-cp38-win_amd64.whl (172.4 MB)
|████████████████████████████████| 172.4 MB 6.4 MB/s
Collecting torchvision==0.15.2
Downloading torchvision-0.15.2-cp38-cp38-win_amd64.whl (1.2 MB)
|████████████████████████████████| 1.2 MB 6.4 MB/s
ERROR: Could not find a version that satisfies the requirement pytorch-cuda==11.8 (from versions: none)
ERROR: No matching distribution found for pytorch-cuda==11.8

Using older version of torch at least the installation worked:

pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118

Now I have this error trying to run the Gradio demo though:

(venv) Q:\PhotoMaker>python gradio_demo/app.py
Traceback (most recent call last):
File "gradio_demo/app.py", line 10, in
import spaces
File "Q:\PhotoMaker\venv\lib\site-packages\spaces_init_.py", line 10, in
from .zero.decorator import GPU
File "Q:\PhotoMaker\venv\lib\site-packages\spaces\zero\decorator.py", line 18, in
from .wrappers import regular_function_wrapper
File "Q:\PhotoMaker\venv\lib\site-packages\spaces\zero\wrappers.py", line 42, in
Process = multiprocessing.get_context('fork').Process
File "C:\Users\Kallamamran\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 239, in get_context
return super().get_context(method)
File "C:\Users\Kallamamran\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 193, in get_context
raise ValueError('cannot find context for %r' % method) from None
ValueError: cannot find context for 'fork'

I wonder what's wrong :(

python3 gradio_demo/aap.py

python=3.10

pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118

Error as flower:
Traceback (most recent call last):
File "/home/vb/PhotoMaker-main/gradio_demo/app.py", line 13, in
from photomaker import PhotoMakerStableDiffusionXLPipeline
ImportError: cannot import name 'PhotoMakerStableDiffusionXLPipeline' from 'photomaker' (/home/vb/anaconda3/envs/photomaker/lib/python3.10/site-packages/photomaker/init.py)

Thanks!

Why is it possible to change to GPU when most calculations use CPU?

I get CUDA out of memory messages on my RTX 3060 12gb.

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.00 GiB (GPU 0; 11.76 GiB total capacity; 8.63 GiB already allocated; 1.37 GiB free; 9.31 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Is it possible run it on 12 Gb VRAM?

I'm waiting for release code !

I have a question about is this technology support controlnet or img2img?

thank you !

Just as an idea, there is an opportunity to improve processing performance if anyone has experience writing code using GPUs. (Parallel work). I heard that processing was done via GPU for another project that processes video, also via Python.

https://github.com/pichugin-i/roop Here is an example of using threads for the GPU.

把所有模型都统一放到百度网盘

比如模型都放 release_model 文件夹

用户下载到源码根目录 就能用了

The PhotoMaker seems using similar pipeline versus IP-Adapter to inject extra image semantic. The PhotoMaker uses special processing for reference image and text embedding to achieve better face swap.

Hi folks! The example notebooks [require]:(https://github.com/TencentARC/PhotoMaker/blob/main/photomaker_demo.ipynb)

from photomaker.pipeline import PhotoMakerStableDiffusionXLPipeline

Installing from the repository doesn't work:

!pip install git+https://github.com/TencentARC/PhotoMaker.git

ERROR: git+https://github.com/TencentARC/PhotoMaker.git does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

I guess the solution is to just download the repo and use PYTHONPATH?

But if so, what about the bin files? E.g. I run this from HuggingFace:

from huggingface_hub import hf_hub_download
photomaker_ckpt = hf_hub_download(repo_id="TencentARC/PhotoMaker", filename="photomaker-v1.bin", repo_type="model")

but I get this error:

401 Client Error: Unauthorized for url:
https://huggingface.co/release_model/resolve/main/photomaker-v1.bin

There were various errors reported in the process, but they have all been resolved. After startup, except for the prompt words, everything else is the default. It's been over half an hour and still no results.

The task manager shows that the graphics card usage is 0, but the memory is always at a high level.