Comments (2)
songweige
commented on August 30, 2024
Hi Mert! Thank you for your interests!
1-) The data.py file supports different formats of videos, say frames in image format or videos in video format. The simplest to include text part instead of adapting the entire CoinRun dataloader, which is very specially designed for MUGEN dataset, is to use this preprocess function after load paired text for each video.
2-) You can follow the instruction in the train section. Basically first train a VQGAN and then a transformer.
3-) Personally I don't recommend training with your own deceive since the training may mess it up. In addition, most of the models are trained with 8 V100 for a larger batch size, which is often quite important.
from tats.
MertEnesYurtseven
commented on August 30, 2024
Thank you for your reply
from tats.
Related Issues (20)
- Cannot open saved model parameters for MUGEN HOT 4
- where to get 'i3d_pretrained_400.pt' & code for IS evaluation HOT 1
- What about the text-to-video generation performance if additional VQVAE is applied on text tokens? HOT 1
- Mugen checkpoint HOT 4
- audio-video generation on custom dataset HOT 8
- Way to future generate frames(infer) conditioned one or a few initial frames of a video sequence. HOT 2
- On the Interpolation Transformer HOT 3
- FileNotFoundError: [Errno 2] No such file or directory: 'datasets/UCF-101/train/metadata_16.pkl' HOT 1
- Where is the codebook loss HOT 1
- Training on FaceForensics HOT 4
- Question about training procedure when dataset is devided with train/test set. HOT 2
- how to load GPT checkpoint? HOT 1
- The Results about Audio to Video Generation Using AudioSet-Drum.
- Unavailable MUGEN checkpoints HOT 2
- Downsample and Upsample ratio for T dimension HOT 1
- Loss starts to diverge When the discriminator opens HOT 5
- Question on training the transformer HOT 5
- Code issues in CausalSelfAttention module
- VQVAE does not appear to be completely Time-Agnostic
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tats.