Comments (3)
Type I solemnly swear that I am up to no good
at the beginning of file name.
Well, jokes aside, I bet logo strongly differs from the rest of hoodie. So, model expects it to be a hole or eye or some sort of relief.
So, there are two ways for you: either train InstantMesh to ignore logos or use StableDiffusion to inpaint cloth over logos. Of course, with SD you'll need to either mask the inpaint zone (if it's common for all images) or use ControlNet\UnCLIP to automatically locate and mask logos for every image.
from instantmesh.
@iiiCpu Is there any possibility to improve InstantMesh output objects by training them? If so, how would you train?
from instantmesh.
@iiiCpu Is there any possibility to improve InstantMesh output objects by training them? If so, how would you train?
Note that i'm not from the developer team. Neither I have experience training exactly this model.
First, InstantMesh launches Zero123++ model to generate initial images of object from different angles. Then InstantMesh uses this images to generate a cloud of points. Finally, it unites this cloud into a final mesh to use in your common 3D engine.
So, first, you need to find out, which step produces an error.
- If Zero123++ is the source (i.e. multiview image has visible defects), you'll need to train it using 3D model of a hoodie. Better to be many different models. Or, at least, one model with different poses and textures. And descriptions on Objaverse syntax. Then you'll need to use this config for training, just edit it so it would use your dataset instead of Objaverse.
- It is possible to train Zero123 purely with images of the object, but it's quite tricky. You'll need many images of the same object on the white background, angles of camera and distanced from the object. Then you'll need to generate description like in this script
- If InstantMesh is the source, you'll need multiple views of the object and it's 3D-model. Use this config to train the model.
Either way, you'll need beefy GPU to train the model. As it feels itself not quite comfortable on 10Gb 3080 RTX during generation, one might expect to need at least 24 Gb 3090 RTX to have a chance for successful training. The more the better.
Oh, you might also try and change base model from Zero123++ to Zero123xl or Stable-Zero123. But as they are slightly different from each other, you'll need to adjust the code base. Or you may just sit here waiting for @TencentARC to release newer version with this support built-in.
from instantmesh.
Related Issues (20)
- IndexError: too many indices for tensor of dimension 2
- Using other multi view models? HOT 1
- Getting errors when running the demo.
- Improve texture quality HOT 2
- Train on nerf of stage 1 with custom dataset HOT 8
- Number of epochs before seeing a decrease in the loss HOT 5
- Fine-Tuning Zero123++ pre-defined range of query image. HOT 1
- Custom Dataset preparing for training HOT 1
- cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' HOT 4
- Out of Memory During Training HOT 6
- Docker - Unexpected error from cudaGetDeviceCount()
- ImportError: cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' HOT 1
- GPU not used for diffusion?
- ninja: build stopped: subcommand failed.
- soldier
- 3 d image
- Support fbx export
- can supports texmap in run app.py ?
- Out of memory running the gradio example
- 训练配置
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from instantmesh.