Comments (2)
@dixiyao Thank you for recognizing and paying attention to our work! With regard to the issue you raised, I will attempt to address it from the following perspectives:
(1) Firstly, in Table 1, we compared results on aligned face datasets (such as FFHQ and CelebA-HQ). The unconditional diffusion model we used was also trained on these datasets, so the size and diversity of the training set may not be as good as datasets like COCO, which could partially explain why the FID values show large differences compared to the COCO dataset.
(2) We only presented examples of stylization and face-swapping using models related to latent diffusion (such as stable diffusion and ControlNet) because we found that these experimental settings (such as learning rate settings) are relatively easy to set and can yield satisfactory results. However, it is more difficult to handle fine-grained conditional information such as sketches in stable diffusion.
(3) We fully agree with your point that if we have control methods that do not require training, why should we use training-required methods like ControlNet? However, my current view is that although training-free methods have significant efficiency advantages, they also have significant limitations (such as the difficulty of using FreeDoM to control sketch conditions in stable diffusion, as mentioned in (2)). Therefore, I believe that the future trend may be a collaboration between training-free and training-required methods. For example, ControlNet can be used to precisely control sketch conditions, and FreeDoM can be used to efficiently control style information (since we were pleasantly surprised by the experimental results in this aspect). I think this is a reasonable expectation, which also indicates that our efforts and attempts in FreeDoM are valuable.
Finally, I hope these answers can help address your questions to some extent!
from freedom.
Thanks very much for your patient answering! I completely agree your view of combining training-free and training-required methods. I think both methods are task specific in some extent. For example, for tasks like face-swapping, if I want to train a condition with Elon Musk's face, I may need N training samples of his face. Though I have tried that ControlNet can achieve relatively good performance even with 100 training samples, it is very have to get 100 images of a single person's face. I'm not sure if my understanding is correct. But if that is the case, training-free shall be a better choice. While as you have said, for some tasks, we should use training-required methods.
from freedom.
Related Issues (19)
- What is the minimum VRAM required to run this? HOT 1
- code release HOT 1
- Question on Eq. 4 HOT 4
- A question on Eq.(1) in the paper HOT 1
- wired output HOT 1
- Plan on releasing code HOT 1
- hyper-parameters used in algorithm 2 HOT 1
- Is this only for 1.4 or is there a 1.5 version? HOT 1
- colab demo
- Is there an example how to use facial landmarks or human keypoints? HOT 2
- Could you please tell me why clip is used to calculate gram matrix instead of VGG.
- some question about coefficient "rho" HOT 1
- error in loading ckpt file
- error when running styled image conditioning demo HOT 1
- train code controlnet + face 有计划开源吗?
- Error(s) in loading state_dict for ControlLDM HOT 2
- 运行SD_style下的txt2img文件时报错,这个模型无法加载
- what does eta = 0.5 mean?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from freedom.