demo_trim2.mp4
- Linux: Ubuntu>18.04
conda create -n lucid python=3.9
conda activate lucid
pip install peft diffusers scipy numpy imageio[ffmpeg] opencv-python Pillow open3d torchvision gradio
pip install torch==2.0.1 timm==0.6.7 # ZoeDepth
pip install plyfile==0.8.1 # Gaussian splatting
cd submodules/depth-diff-gaussian-rasterization-min
python setup.py install
cd ../simple-knn
python setup.py install
cd ../..
# Default Example
python run.py
To run with your own inputs and prompts, attach following arguments after run.py
.
-img
: path of input image.-t
: text prompt. Can be either path to txt file or the text iteslf.-nt
: negative text prompt. Can be either path to txt file or the text iteslf.-cg
: camera extrinsic path for generating scenes. Can be one of "Rotate_360", "LookAround", or "LookDown".-cr
: camera extrinsic path for rendering videos. Can be one of "Back_and_forth", "LLFF", or "Headbanging".--seed
: manual seed for Stable Diffusion inpainting.--diff_steps
: number of denoising steps for Stable Diffusion inpainting. Default is 50.-s
: path to save results.
- โ December 7, 2023: Colab implementation is now available thanks to @camenduru!
- โ December 6, 2023: Code release!
- โ November 22, 2023: We have released our paper, LucidDreamer on arXiv.
Please cite us if you find our project useful!
@article{chung2023luciddreamer,
title={LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes},
author={Chung, Jaeyoung and Lee, Suyoung and Nam, Hyeongjin and Lee, Jaerin and Lee, Kyoung Mu},
journal={arXiv preprint arXiv:2311.13384},
year={2023}
}
We deeply appreciate ZoeDepth, Stability AI, and Runway for their models.
If you have any questions, please email [email protected]
, [email protected]
.