Comments (3)
Thanks for your wonderful work! I am interested in applying autoregressive to achieve a length-flexible output. Could this be implemented by changing the way the model infers, like the LLMs?
Thanks for your interest. What inference algorithm of LLM are you referring to specifically?
from latte.
I mean, generating subsequent frames using the previous frames as input (and perhaps adding a special end token?), instead of generating 16 frames at once. Thus we can accept training videos with any length, and generate longer and more length-flexible videos.
from latte.
I mean, generating subsequent frames using the previous frames as input (and perhaps adding a special end token?), instead of generating 16 frames at once. Thus we can accept training videos with any length, and generate longer and more length-flexible videos.
Not sure about performance, since the model was trained directly on 16 frames of video. You can try it, and if there are better results, welcome PR.
from latte.
Related Issues (20)
- Train code of t2v? HOT 2
- 如何复现主页展示的t2v效果? HOT 3
- t2v只支持16帧吗?我改成更多比如32帧就啥都看不到了 HOT 1
- Implementation of compression frame patch embedding (Fig. 3b) HOT 2
- diffusion noise modify HOT 1
- About Training Speed HOT 3
- About resume checkpoint HOT 2
- Extra key in ucf101.pt HOT 7
- Why choose these datasets and why not compare with pika, SVD or Gen2? HOT 1
- trained and sample result very strange (我自己训练复现的效果很奇怪) HOT 21
- What is the difference between Latte and ViViT? HOT 2
- RuntimeError: "LayerNormKernelImpl" not implemented for 'Half' HOT 2
- RuntimeError: "slow_conv2d_cpu" not implemented for 'Half' HOT 2
- image_size = [256,512] HOT 4
- CUDA out of memory HOT 4
- Evaluate the FVD? HOT 5
- Some weights of AutoencoderKL were not initialized from the model checkpoint at /path/to/Latte/t2v_required_models/ and are newly initialized because the shapes did not match: HOT 3
- FaceForensics数据集 HOT 3
- No positional embeddings in LatteT2V?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from latte.