Giter Club home page Giter Club logo

nyu-supergan's Introduction

SuperGAN

LICENSE PYTHON PYTORCH CUDA


๐Ÿ”ง Dependencies and Installation

Dependencies

Installation

  1. Clone repo

    git clone https://github.com/95anantsingh/NYU-SuperGAN.git
    cd NYU-SuperGAN
  2. Create conda environment

    conda env create -f environment.yml
  3. Download weights

    wget -i SuperGAN/weight_urls --directory-prefix SuperGAN/data/weights

โšก Quick Inference

cd SuperGAN
conda activate SuperGAN
python faceswap.py

Now play data/output/output.mp4 to see results
Make necessary changes in faceswap.py for different input videos.


๐Ÿ“ Project Structure

  1. SuperGAN contains all the project files
  2. SuperGAN/data contains input output videos and pretrained weights
  3. SuperGAN/faceswap.py main inference file

๐Ÿ“˜ Documentation

Project report can be found at docs/SuperGAN_Report.pdf

Ideal setup is with NVIDIA Quadro RTX 8000 (48 GB GDDR6) on Linux


๐Ÿ“œ License

SuperGAN is released under Apache License Version 2.0.


๐Ÿ“ง Contact

If you have any question, please email [email protected]

This Project was part of graduate level Deep Learning course at New York University

nyu-supergan's People

Contributors

95anantsingh avatar n0vay avatar stutibiyani avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

nyu-supergan's Issues

Looks like inference is not working

/content/NYU-SuperGAN/SuperGAN
Downloading: "https://download.pytorch.org/models/resnet152-394f9c45.pth" to /root/.cache/torch/hub/checkpoints/resnet152-394f9c45.pth
100% 230M/230M [00:06<00:00, 35.6MB/s]
=> using GPU devices: 0
=> Loading face pose model: "hopenet_robust_alpha1.pth"...
=> Loading face landmarks model: "hr18_wflw_landmarks.pth"...
=> Loading face segmentation model: "celeba_unet_256_1_2_segmentation_v2.pth"...
=> Loading face reenactment model: "nfv_msrunet_256_1_2_reenactment_v2.1.pth"...
=> Loading face completion model: "ijbc_msrunet_256_1_2_inpainting_v2.pth"...
=> Loading face blending model: "ijbc_msrunet_256_1_2_blending_v2.pth"...
=> Loading face restoring model:"GFPGANCleanv1-NoCE-C2.pth"...
Downloading: "https://github.com/xinntao/facexlib/releases/download/v0.1.0/detection_Resnet50_Final.pth" to /usr/local/lib/python3.7/dist-packages/facexlib/weights/detection_Resnet50_Final.pth

100% 104M/104M [00:10<00:00, 10.4MB/s]
=> Detecting faces in video: "shinzo_abe.mp4..."
100% 600/600 [11:40<00:00,  1.17s/frames]
=> Extracting sequences from detections in video: "shinzo_abe.mp4"...
100% 601/601 [00:00<00:00, 11473.57it/s]
=> Cropping video sequences from video: "shinzo_abe.mp4"...
100% 600/600 [00:03<00:00, 155.42it/s]
=> Computing face poses for video: "shinzo_abe_seq00.mp4"...
100% 5/5 [00:04<00:00,  1.14batches/s]
=> Computing face landmarks for video: "shinzo_abe_seq00.mp4"...
100% 10/10 [00:10<00:00,  1.06s/batches]
=> Computing face segmentation for video: "shinzo_abe_seq00.mp4"...
  0% 0/10 [00:00<?, ?batches/s]
Traceback (most recent call last):
  File "faceswap.py", line 76, in <module>
    select_source, select_target, finetune)
  File "/content/NYU-SuperGAN/SuperGAN/fsgan/inference/swap.py", line 283, in __call__
    source_cache_dir, source_seq_file_path, _ = self.cache(source_path)
  File "/content/NYU-SuperGAN/SuperGAN/fsgan/preprocess/preprocess_video.py", line 480, in cache
    self.process_segmentation(input_path, output_dir, seq_file_path)
  File "/content/NYU-SuperGAN/SuperGAN/fsgan/preprocess/preprocess_video.py", line 384, in process_segmentation
    raw_segmentation = self.S(frame)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/NYU-SuperGAN/SuperGAN/fsgan/models/simple_unet_02.py", line 71, in forward
    up4 = self.up_concat4(conv4, center)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/NYU-SuperGAN/SuperGAN/fsgan/models/simple_unet_02.py", line 135, in forward
    outputs2 = self.conv1d(outputs2,)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py", line 302, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py", line 299, in _conv_forward
    self.padding, self.dilation, self.groups)
RuntimeError: Expected 2D (unbatched) or 3D (batched) input to conv1d, but got input of size: [64, 1024, 32, 32]

Error Getting Environment Set Up on Windows 10

Hello,

This looks like a very exciting piece of work and I am looking forward to giving it a whirl. However, when attempting to set up the environment using environment.yml in anaconda, I am running into an error.

Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:

  • pyyaml==6.0=py38h0a891b7_4
  • brotlipy==0.7.0=py38h0a891b7_1004
  • nettle==3.6=he412f7d_0
  • argon2-cffi-bindings==21.2.0=py38h0a891b7_2
  • libwebp==1.2.2=h3452ae3_0
  • graphite2==1.3.13=h58526e2_1001
  • libdrm==2.4.109=h7f98852_0
  • cryptography==37.0.1=py38h9ce1e76_0
  • zeromq==4.3.4=h9c3ff4c_1
  • debugpy==1.6.0=py38hfa26641_0
  • pysocks==1.7.1=py38h578d9bd_5
  • ca-certificates==2021.10.8=ha878542_0
  • xorg-libsm==1.2.3=hd9c2040_1000
  • c-ares==1.18.1=h7f98852_0
  • libglib==2.70.2=h174f98d_4
  • numpy-base==1.21.5=py38hf524024_2
  • brotli-bin==1.0.9=h166bdaf_7
  • libgomp==11.2.0=h1d223b6_16
  • qt==5.12.9=h1304e3e_6
  • lame==3.100=h7f98852_1001
  • x265==3.5=h924138e_3
  • giflib==5.2.1=h36c2ea0_2
  • xorg-fixesproto==5.0=h7f98852_1002
  • xorg-libxext==1.3.4=h7f98852_1
  • xorg-libxi==1.7.10=h7f98852_0
  • jasper==2.0.33=ha77e612_0
  • ipykernel==6.13.0=py38h7f3c49e_0
  • xorg-libice==1.0.10=h7f98852_0
  • matplotlib-base==3.5.2=py38h826bfd8_0
  • dbus==1.13.6=h5008d03_3
  • xorg-xproto==7.0.31=h7f98852_1007
  • libopencv==4.5.5=py38h2380011_9
  • libblas==3.9.0=12_linux64_mkl
  • libbrotlidec==1.0.9=h166bdaf_7
  • importlib-metadata==4.11.3=py38h578d9bd_1
  • freeglut==3.2.2=h9c3ff4c_1
  • krb5==1.19.3=h08a2579_0
  • certifi==2021.10.8=py38h578d9bd_2
  • mkl-service==2.4.0=py38h95df7f1_0
  • terminado==0.13.3=py38h578d9bd_1
  • libogg==1.3.4=h7f98852_1
  • harfbuzz==4.2.0=h40b6f09_0
  • openjpeg==2.4.0=hb52868f_1
  • expat==2.4.8=h27087fc_0
  • libnsl==2.0.0=h7f98852_0
  • libsodium==1.0.18=h36c2ea0_1
  • keyutils==1.6.1=h166bdaf_0
  • intel-openmp==2021.4.0=h06a4308_3561
  • setuptools==62.1.0=py38h578d9bd_0
  • pandoc==2.18=ha770c72_0
  • libstdcxx-ng==11.2.0=he4da1e4_16
  • libtiff==4.3.0=h542a066_3
  • pyrsistent==0.18.1=py38h0a891b7_1
  • libprotobuf==3.20.1=h6239696_0
  • liblapacke==3.9.0=12_linux64_mkl
  • unicodedata2==14.0.0=py38h0a891b7_1
  • pytorch==1.10.2=py3.8_cuda11.3_cudnn8.2.0_0
  • widgetsnbextension==3.6.0=py38h578d9bd_0
  • xz==5.2.5=h516909a_1
  • aom==3.3.0=h27087fc_1
  • fontconfig==2.14.0=h8e229c2_0
  • libglu==9.0.0=he1b5a44_1001
  • libcblas==3.9.0=12_linux64_mkl
  • gst-plugins-base==1.20.2=hcf0ee16_0
  • jbig==2.1=h7f98852_2003
  • gmp==6.2.1=h58526e2_0
  • ncurses==6.3=h27087fc_1
  • cudatoolkit==11.3.1=h2bc3f7f_2
  • openssl==3.0.3=h166bdaf_0
  • libclang==13.0.1=default_hc23dcda_0
  • mistune==0.8.4=py38h497a2fe_1005
  • svt-av1==0.9.1=h27087fc_0
  • yaml==0.2.5=h7f98852_2
  • libxml2==2.9.12=h885dcf4_1
  • icu==69.1=h9c3ff4c_0
  • lcms2==2.12=hddcbb42_0
  • libdeflate==1.10=h7f98852_0
  • xorg-libxrender==0.9.10=h7f98852_1003
  • cffi==1.15.0=py38h3931269_0
  • libev==4.33=h516909a_1
  • libopus==1.3.1=h7f98852_1
  • libedit==3.1.20191231=he28a2e2_2
  • ipython==8.3.0=py38h578d9bd_0
  • tornado==6.1=py38h0a891b7_3
  • libpng==1.6.37=h21135ba_2
  • kiwisolver==1.4.2=py38h43d8883_1
  • _openmp_mutex==4.5=2_gnu
  • zstd==1.5.2=ha95c52a_0
  • pixman==0.40.0=h36c2ea0_0
  • libcurl==7.83.0=h2283fc2_0
  • alsa-lib==1.2.3.2=h166bdaf_0
  • mysql-common==8.0.29=h26416b9_0
  • libvpx==1.11.0=h9c3ff4c_3
  • libnghttp2==1.47.0=he49606f_0
  • mkl_random==1.2.2=py38h1abd341_0
  • markupsafe==2.1.1=py38h0a891b7_1
  • pyqtchart==5.12=py38h7400c14_8
  • libxcb==1.13=h7f98852_1004
  • libiconv==1.16=h516909a_0
  • gstreamer==1.20.2=hd4edc92_0
  • libssh2==1.10.0=ha35d2d1_2
  • libuv==1.43.0=h7f98852_0
  • libvorbis==1.3.7=h9c3ff4c_0
  • mysql-libs==8.0.29=hbc51c84_0
  • ffmpeg==4.4.1=hd7ab26d_2
  • cairo==1.16.0=ha12eb4b_1010
  • mkl_fft==1.3.1=py38h8666266_1
  • pyqt==5.12.3=py38h578d9bd_8
  • xorg-xextproto==7.3.0=h7f98852_1002
  • scipy==1.8.0=py38h56a6a73_1
  • xorg-libxdmcp==1.1.3=h7f98852_0
  • pthread-stubs==0.4=h36c2ea0_1001
  • py-opencv==4.5.5=py38h7f3c49e_9
  • lerc==3.0=h9c3ff4c_0
  • gnutls==3.6.13=h85f3911_1
  • libzlib==1.2.11=h166bdaf_1014
  • xorg-libxfixes==5.0.3=h7f98852_1004
  • sqlite==3.38.5=h4ff8645_0
  • liblapack==3.9.0=12_linux64_mkl
  • libwebp-base==1.2.2=h7f98852_1
  • libllvm13==13.0.1=hf817b99_2
  • xorg-libx11==1.7.2=h7f98852_0
  • jupyter_core==4.9.2=py38h578d9bd_0
  • libgfortran5==11.2.0=h5c6108e_16
  • xorg-kbproto==1.0.7=h7f98852_1002
  • gettext==0.19.8.1=h73d1719_1008
  • openh264==2.1.1=h780b84a_0
  • matplotlib==3.5.2=py38h578d9bd_0
  • xorg-libxau==1.0.9=h7f98852_0
  • psutil==5.9.0=py38h0a891b7_1
  • nss==3.77=h2350873_0
  • xorg-inputproto==2.3.2=h7f98852_1002
  • libevent==2.1.10=h28343ad_4
  • nspr==4.32=h9c3ff4c_1
  • zlib==1.2.11=h166bdaf_1014
  • libgcc-ng==11.2.0=h1d223b6_16
  • bzip2==1.0.8=h7f98852_4
  • ld_impl_linux-64==2.36.1=hea4e1c9_2
  • freetype==2.10.4=h0708190_1
  • hdf5==1.12.1=nompi_h4df4325_104
  • pyqt5-sip==4.19.18=py38h709712a_8
  • fonttools==4.33.3=py38h0a891b7_0
  • _libgcc_mutex==0.1=conda_forge
  • brotli==1.0.9=h166bdaf_7
  • jedi==0.18.1=py38h578d9bd_1
  • libpciaccess==0.16=h516909a_0
  • libpq==14.2=h676c864_0
  • pillow==9.1.0=py38h0ee0e06_2
  • tk==8.6.12=h27826a3_0
  • numpy==1.21.5=py38he7a7128_2
  • lz4-c==1.9.3=h9c3ff4c_1
  • python==3.8.13=ha86cf86_0_cpython
  • libva==2.14.0=h7f98852_0
  • mkl==2021.4.0=h06a4308_640
  • pyzmq==22.3.0=py38hfc09fa9_2
  • libuuid==2.32.1=h7f98852_1000
  • libffi==3.4.2=h7f98852_5
  • opencv==4.5.5=py38h578d9bd_9
  • libbrotlicommon==1.0.9=h166bdaf_7
  • libbrotlienc==1.0.9=h166bdaf_7
  • libgfortran-ng==11.2.0=h69a702a_16
  • pyqtwebengine==5.12.1=py38h7400c14_8
  • readline==8.1=h46c0cb4_0
  • xorg-renderproto==0.11.1=h7f98852_1002
  • libxkbcommon==1.0.3=he3ba5ed_0
  • pyqt-impl==5.12.3=py38h0ffb2e6_8
  • future==0.18.2=py38h578d9bd_5
  • jpeg==9e=h166bdaf_1
  • pcre==8.45=h9c3ff4c_0
  • x264==1!161.3030=h7f98852_1

Any and all assistance much welcomed!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.