afruehstueck / vive3d Goto Github PK
View Code? Open in Web Editor NEWOfficial Code of CVPR 2023 Paper "VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs"
License: Other
Official Code of CVPR 2023 Paper "VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs"
License: Other
after running it reported that :
No such file or directory {config.HOME}/projects/STIT/pretrained_models/79999_iter.pth
Seems this path is defined in segmenter.py,so how to get it?
Hi,
Thank you for the brilliant work!
When I was trying to run the inversion and re-rendering code on some additional video sequences, I obtained very bad results:
Basically, during fitting I followed the given code and selected 7 frames out of a 600 frames video for personalized fitting (step 1), then I run invert for all frames in the video. I modified edit_video.py
file to first disable w editing -- so I just pass original w_person
and loaded w_offsets_video
to generator.generate()
method, which is modified to do the below:
def generate(self, w, yaw=0.0, pitch=0.0, focal_length=None, noise_mode='const', output_all=False, grad=False, w_offset=None,):
# convert angles to camera parameter(s)
camera_params = self.get_camera_parameters(yaw=yaw, pitch=pitch, focal_length=focal_length)
# convert w to correct size
# (,512) -> (1, 512)
if len(w.shape) == 1:
w = w.unsqueeze(0)
# (x, 512) -> (x, 1, 512)
if len(w.shape) == 2:
w = w.unsqueeze(1)
# repeat to (1, 14, 512)
if w.shape[1] == 1:
w = w.repeat(1, self.active_G.backbone.synthesis.num_ws, 1)
num_images = max(len(w), len(camera_params))
# replicate if single w was passed
if w.shape[0] < num_images:
w = w.repeat(num_images, 1, 1)
# replicate if single camera parameter was passed
if camera_params.shape[0] < num_images:
camera_params = camera_params.repeat(num_images, 1)
if w_offset is not None:
w = w + w_offset
assert w.shape[0] == camera_params.shape[0], f'incompatible size of w ({w.shape}) and camera params ({camera_params.shape})'
images = []
images_raw = []
depths = []
self.active_G.to(self.device)
I.e., it adds w_offset
back to duplicated w
.
May I ask if the result is expected, or did I make a mistake in implementing the re-rendering function?
Many thanks!
It leads to another paper: https://arxiv.org/abs/2203.07293
Hi,
Congrats on this great research. The video demo shows the head exchange capabilities, but I can't seem to find any code or documentation about how that is done. Is there a Jupyter notebook that you could upload for that demo?
thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.