Neural Style and MSG-Net

Home Page: http://hangzh.com/PyTorch-Style-Transfer/

License: MIT License

Python 3.23% Shell 0.04% Jupyter Notebook 96.72% Dockerfile 0.02%

style-transfer deep-neural-networks real-time

pytorch-multi-style-transfer's Introduction

PyTorch-Style-Transfer

This repo provides PyTorch Implementation of MSG-Net (ours) and Neural Style (Gatys et al. CVPR 2016), which has been included by ModelDepot. We also provide Torch implementation and MXNet implementation.

Tabe of content

Real-time Style Transfer using MSG-Net
- Stylize Images using Pre-trained Model
- Train Your Own MSG-Net Model
Slow Neural Style Transfer

MSG-Net

Multi-style Generative Network for Real-time Transfer [arXiv] [project]
Hang Zhang, Kristin Dana

@article{zhang2017multistyle,
	title={Multi-style Generative Network for Real-time Transfer},
	author={Zhang, Hang and Dana, Kristin},
	journal={arXiv preprint arXiv:1703.06953},
	year={2017}
}

Stylize Images Using Pre-trained MSG-Net

Download the pre-trained model

git clone [email protected]:zhanghang1989/PyTorch-Style-Transfer.git
cd PyTorch-Style-Transfer/experiments
bash models/download_model.sh

Camera Demo

python camera_demo.py demo --model models/21styles.model

Test the model

python main.py eval --content-image images/content/venice-boat.jpg --style-image images/21styles/candy.jpg --model models/21styles.model --content-size 1024

If you don't have a GPU, simply set --cuda=0. For a different style, set --style-image path/to/style. If you would to stylize your own photo, change the --content-image path/to/your/photo. More options:
- --content-image: path to content image you want to stylize.
- --style-image: path to style image (typically covered during the training).
- --model: path to the pre-trained model to be used for stylizing the image.
- --output-image: path for saving the output image.
- --content-size: the content image size to test on.
- --cuda: set it to 1 for running on GPU, 0 for CPU.

Train Your Own MSG-Net Model

Download the COCO dataset
```
bash dataset/download_dataset.sh
```
Train the model
```
python main.py train --epochs 4
```

If you would like to customize styles, set --style-folder path/to/your/styles. More options:
- --style-folder: path to the folder style images.
- --vgg-model-dir: path to folder where the vgg model will be downloaded.
- --save-model-dir: path to folder where trained model will be saved.
- --cuda: set it to 1 for running on GPU, 0 for CPU.

Neural Style

Image Style Transfer Using Convolutional Neural Networks by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge.

python main.py optim --content-image images/content/venice-boat.jpg --style-image images/21styles/candy.jpg

--content-image: path to content image.
--style-image: path to style image.
--output-image: path for saving the output image.
--content-size: the content image size to test on.
--style-size: the style image size to test on.
--cuda: set it to 1 for running on GPU, 0 for CPU.

Acknowledgement

The code benefits from outstanding prior work and their implementations including:

Texture Networks: Feed-forward Synthesis of Textures and Stylized Images by Ulyanov et al. ICML 2016. (code)
Perceptual Losses for Real-Time Style Transfer and Super-Resolution by Johnson et al. ECCV 2016 (code) and its pytorch implementation code by Abhishek.
Image Style Transfer Using Convolutional Neural Networks by Gatys et al. CVPR 2016 and its torch implementation code by Johnson.

pytorch-multi-style-transfer's People

Contributors

Stargazers

Watchers

Forkers

ksharpdabu parneetk chunniunai220ml xibinyue zencyyoung merczz vyraun ml-lab styleflow computervisionrutgers ziky90 handcraftsman hli1221 clcarwin 2prime liuguoyou connor323 qiaott zetian shubhampachori12110095 cheetah132 blitu12345 nstep boogh jvision alvincjin whaozl idgmatrix riktimmondal leix8 trexwithoutt selimam ddl0 liaoheping achaiah aust-hansen baifengbai mrashes backnotprop shosinnsha ruijin1988 pandinosaurus wutianyirosun loicem barbecacov shuizhilinxin pinglmlcv zhouhang95 makesaltlake huangjundashuaige afcarl hologerry yingweiy gstrezoski deepbrain-ml jhlegarreta zgaoooo zhulei2016 yongxiongwei karenerobinson lanfeng12 dip-styletransfer danieltao gabeochieng ltumorsegmentationtest abbyyangbb rjanser mryimings tramper2 sgrpwr jmnie wellenwoo paintthatagain duweihua yangsn0719 globz-eu slinn650 ryotamono ayanlola2002 zoehedgehog lusonpan62678 ycjing lwpyh ankushswar1 albert-8 faizwhb marcoleewow jarygrace sergeman ritesh2212 jcpan7 alanhzh cyberfork freegliboracle kmes055 taroorat cat-l zhiyuanh suchun-sv kknet

pytorch-multi-style-transfer's Issues

Error when running style_model.load_state_dict(torch.load('21styles.model'), False)

When using Python 3 and PyTorch commit 7f130c8494d235054917a7abfcbe1058a65d628a
upon running the line of code style_model.load_state_dict(torch.load('21styles.model'), False)
get the errors below. If one sets instancenorm to have track_running_stats=True then the number of features needs to be explicitly input:
https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/instancenorm.py

Traceback (most recent call last):
File "msgnet.py", line 229, in
style_model.load_state_dict(torch.load('21styles.model'), False)
File "/home/benson/SmallProjects/Picture/virtualenvpytorch/virtualenvpytorch/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 837, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Net:
Unexpected running stats buffer(s) "model1.1.running_mean" and "model1.1.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model1.3.conv_block.0.running_mean" and "model1.3.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model1.3.conv_block.3.running_mean" and "model1.3.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model1.3.conv_block.6.running_mean" and "model1.3.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model1.4.conv_block.0.running_mean" and "model1.4.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model1.4.conv_block.3.running_mean" and "model1.4.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model1.4.conv_block.6.running_mean" and "model1.4.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.0.1.running_mean" and "model.0.1.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.0.3.conv_block.0.running_mean" and "model.0.3.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.0.3.conv_block.3.running_mean" and "model.0.3.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.0.3.conv_block.6.running_mean" and "model.0.3.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.0.4.conv_block.0.running_mean" and "model.0.4.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.0.4.conv_block.3.running_mean" and "model.0.4.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.0.4.conv_block.6.running_mean" and "model.0.4.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.2.conv_block.0.running_mean" and "model.2.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.2.conv_block.3.running_mean" and "model.2.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.2.conv_block.6.running_mean" and "model.2.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.3.conv_block.0.running_mean" and "model.3.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.3.conv_block.3.running_mean" and "model.3.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.3.conv_block.6.running_mean" and "model.3.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.4.conv_block.0.running_mean" and "model.4.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.4.conv_block.3.running_mean" and "model.4.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.4.conv_block.6.running_mean" and "model.4.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.5.conv_block.0.running_mean" and "model.5.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.5.conv_block.3.running_mean" and "model.5.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.5.conv_block.6.running_mean" and "model.5.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.6.conv_block.0.running_mean" and "model.6.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.6.conv_block.3.running_mean" and "model.6.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.6.conv_block.6.running_mean" and "model.6.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.7.conv_block.0.running_mean" and "model.7.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.7.conv_block.3.running_mean" and "model.7.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.7.conv_block.6.running_mean" and "model.7.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.8.conv_block.0.running_mean" and "model.8.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.8.conv_block.3.running_mean" and "model.8.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.8.conv_block.6.running_mean" and "model.8.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.9.conv_block.0.running_mean" and "model.9.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.9.conv_block.3.running_mean" and "model.9.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.9.conv_block.6.running_mean" and "model.9.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
Unexpected running stats buffer(s) "model.10.running_mean" and "model.10.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.

Error Training TypeError: 'NoneType' object is not callable

I was able to get my environment setup successfully to run eval; however, now, trying train I'm running into an issue. Not sure if it's a syntax issues or if something else is going on? You help is greatly appreciated.


#!/bin/bash
#SBATCH --job-name=train-pytorch
#SBATCH --mail-type=END,FAIL
#SBATCH [email protected]
#SBATCH --ntasks=1
#SBATCH --time=00:10:00
#SBATCH --mem=8000
#SBATCH --gres=gpu:p100:2
#SBATCH --cpus-per-task=6
#SBATCH --output=%x_%j.log
#SBATCH --error=%x_%j.err

source ~/scratch/moldach/PyTorch-Style-Transfer/experiments/venv/bin/activate

python main.py train \
  --epochs 4 \
  --style-folder /scratch/moldach/PyTorch-Style-Transfer/experiments/images/9styles \
  --vgg-model-dir vgg-model/ \
  --save-model-dir checkpoint/

/scratch/moldach/first-order-model/venv/lib/python3.6/site-packages/torchvision/transforms/transforms.py:188: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
Traceback (most recent call last):
  File "main.py", line 295, in <module>
    main()
  File "main.py", line 41, in main
    train(args)
  File "main.py", line 135, in train
    utils.init_vgg16(args.vgg_model_dir)
  File "/scratch/moldach/PyTorch-Style-Transfer/experiments/utils.py", line 102, in init_vgg16
    for (src, dst) in zip(vgglua.parameters()[0], vgg.parameters()):
TypeError: 'NoneType' object is not callable

`pip freeze`:

$ pip freeze
-f /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/nix/avx2
-f /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/nix/generic
-f /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
cffi==1.11.5
cloudpickle==0.5.3
cycler==0.10.0
dask==0.18.2
dataclasses==0.8
decorator==4.4.2
future==0.18.2
imageio==2.9.0
imageio-ffmpeg==0.4.3
kiwisolver==1.3.1
matplotlib==3.3.4
networkx==2.5
numpy==1.19.1
pandas==0.23.4
Pillow==8.1.2
pycparser==2.18
pygit==0.1
pyparsing==2.4.7
python-dateutil==2.8.1
pytz==2018.5
PyWavelets==1.1.1
PyYAML==5.1
scikit-image==0.17.2
scikit-learn==0.19.2
scipy==1.4.1
six==1.15.0
tifffile==2020.9.3
toolz==0.9.0
torch==1.7.0
torchfile==0.1.0
torchvision==0.2.1
tqdm==4.24.0
typing-extensions==3.7.4.3

Update for PyTorch 0.4.0

PyTorch 0.4.0 was released on April 24 and unfortunately the pre-trained weights from before are not compatible.

On the notebook I get

style_model = Net(ngf=128)
style_model.load_state_dict(torch.load('21styles.model'), False)

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-7-ce41c62c2272> in <module>()
      1 style_model = Net(ngf=128)
----> 2 style_model.load_state_dict(torch.load('21styles.model'), False)

/usr/lib/python3.6/site-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
    719         if len(error_msgs) > 0:
    720             raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
--> 721                                self.__class__.__name__, "\n\t".join(error_msgs)))
    722 
    723     def parameters(self):

RuntimeError: Error(s) in loading state_dict for Net:
	Unexpected running stats buffer(s) "model1.1.running_mean" and "model1.1.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model1.3.conv_block.0.running_mean" and "model1.3.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model1.3.conv_block.3.running_mean" and "model1.3.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model1.3.conv_block.6.running_mean" and "model1.3.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model1.4.conv_block.0.running_mean" and "model1.4.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model1.4.conv_block.3.running_mean" and "model1.4.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model1.4.conv_block.6.running_mean" and "model1.4.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.0.1.running_mean" and "model.0.1.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.0.3.conv_block.0.running_mean" and "model.0.3.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.0.3.conv_block.3.running_mean" and "model.0.3.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.0.3.conv_block.6.running_mean" and "model.0.3.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.0.4.conv_block.0.running_mean" and "model.0.4.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.0.4.conv_block.3.running_mean" and "model.0.4.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.0.4.conv_block.6.running_mean" and "model.0.4.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.2.conv_block.0.running_mean" and "model.2.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.2.conv_block.3.running_mean" and "model.2.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.2.conv_block.6.running_mean" and "model.2.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.3.conv_block.0.running_mean" and "model.3.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.3.conv_block.3.running_mean" and "model.3.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.3.conv_block.6.running_mean" and "model.3.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.4.conv_block.0.running_mean" and "model.4.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.4.conv_block.3.running_mean" and "model.4.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.4.conv_block.6.running_mean" and "model.4.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.5.conv_block.0.running_mean" and "model.5.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.5.conv_block.3.running_mean" and "model.5.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.5.conv_block.6.running_mean" and "model.5.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.6.conv_block.0.running_mean" and "model.6.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.6.conv_block.3.running_mean" and "model.6.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.6.conv_block.6.running_mean" and "model.6.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.7.conv_block.0.running_mean" and "model.7.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.7.conv_block.3.running_mean" and "model.7.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.7.conv_block.6.running_mean" and "model.7.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.8.conv_block.0.running_mean" and "model.8.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.8.conv_block.3.running_mean" and "model.8.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.8.conv_block.6.running_mean" and "model.8.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.9.conv_block.0.running_mean" and "model.9.conv_block.0.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.9.conv_block.3.running_mean" and "model.9.conv_block.3.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.9.conv_block.6.running_mean" and "model.9.conv_block.6.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.
	Unexpected running stats buffer(s) "model.10.running_mean" and "model.10.running_var" for InstanceNorm2d with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because InstanceNorm2d does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in InstanceNorm2d to enable them. See the documentation of InstanceNorm2d for details.

struct.error: unpack requires a buffer of 4 bytes

Dear author,
Thank you so much for sharing a useful code.
I able to run your evaluation code, but face the following error during runing of training code:
File "main.py", line 41, in main
train(args)
File "main.py", line 135, in train
utils.init_vgg16(args.vgg_model_dir)
File "/home2/st118370/models/PyTorch-Multi-Style-Transfer/experiments/utils.py", line 100, in init_vgg16
vgglua = load_lua(os.path.join(model_folder, 'vgg16.t7'))
File "/home2/st118370/anaconda3/envs/pytorch-py3/lib/python3.7/site-packages/torchfile.py", line 424, in load
return reader.read_obj()
File "/home2/st118370/anaconda3/envs/pytorch-py3/lib/python3.7/site-packages/torchfile.py", line 310, in read_obj
typeidx = self.read_int()
File "/home2/st118370/anaconda3/envs/pytorch-py3/lib/python3.7/site-packages/torchfile.py", line 277, in read_int
return self._read('i')[0]
File "/home2/st118370/anaconda3/envs/pytorch-py3/lib/python3.7/site-packages/torchfile.py", line 271, in _read
return struct.unpack(fmt, self.f.read(sz))
struct.error: unpack requires a buffer of 4 bytes

how can i resolve this problem? kindly guide. thanks

Unable to resume training

Hey,

So I started training a model, but seeing how long it was going to take I wanted to double check I could successfully resume training.

I ran:
python3 main.py train --epochs 4 --style-folder images/xmas-styles/ --save-model-dir trained_models/
until it generated the first checkpoint, then I ran
python3 main.py train --epochs 4 --style-folder images/xmas-styles/ --save-model-dir trained_models/ --resume trained_models/Epoch_0iters_8000_Sat_Dec__9_18\:10\:43_2017_1.0_5.0.model
and waiting for the first feedback report, which was
Sat Dec 9 18:17:09 2017 Epoch 1: [2000/123287] content: 254020.831359 style: 1666218.549250 total: 1920239.380609
so it appeared to not have resumed at all.

Also slight side question... Say I train with --epochs 4 til I get final model... If I were to use the last checkpoint before final to resume, but set --epochs 5 or higher, would that work correctly and just keep going through to 5 epochs before generating another final, and have no issues etc?

"TypeError: 'torch.FloatTensor' object is not callable" running demo on CPU

Sorry if I'm missing something, I'm unfamiliar with PyTorch.
I'm running the demo on CPU on a Mac and getting the following error:

  File "camera_demo.py", line 93, in <module>
    main()
  File "camera_demo.py", line 90, in main
    run_demo(args, mirror=True)
  File "camera_demo.py", line 60, in run_demo
    simg = style_v.data().numpy()
TypeError: 'torch.FloatTensor' object is not callable

Thanks.

doesn't work anymore

Refuses to split frames, gives this error (colab)
error: OpenCV(4.7.0) /io/opencv/modules/imgcodecs/src/loadsave.cpp:783: error: (-215:Assertion failed) !_img.empty() in function 'imwrite'

cuda runtime error (2) : out of memory - but memory doesn't run out

Thank you for reading this!

I am facing a problem while trying to train my own MSG-Net Model. While I run

$ python main.py train --epochs 1 --cuda 1

after displaying the network structure, I got this error:

RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58

However, after checking the GPU memory of my server, I feel like the memory is enough to train this model (correct me if I am wrong):

$ nvidia-smi
Thu Nov  8 15:11:22 2018       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.130                Driver Version: 384.130                   |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  TITAN Xp            Off  | 00000000:65:00.0 Off |                  N/A |
| 53%   83C    P2   210W / 250W |   7504MiB / 12188MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+

Does anyone have any idea for solving this? Any suggestion is appreciated.
Thanks in advance!

Color produced by eval doesn't match demo

Hi ! Thanks for sharing the code. I've ran the eval program using the defaults provided and I noticed the color tends to be much dimmer than what is shown on the homepage here. Is there something that I am missing? The command I used was

python main.py --style-image ./images/21styles/udnie.jpg --content-image ./images/content/venice-boat.jpg

test error

I got this error.

Traceback (most recent call last):
File "main.py", line 290, in
main()
File "main.py", line 45, in main
evaluate(args)
File "main.py", line 258, in evaluate
output = utils.color_match(output, style_v)
File "/media/hdd1/donghyun/armor/PyTorch-Multi-Style-Transfer/experiments/utils.py", line 150, in color_match
src_flat_nrom_trans = matSqrt(dst_flat_cov_eye) * matSqrt(src_flat_cov_eye).inverse * src_norm
RuntimeError: mul() received an invalid combination of arguments - got (builtin_function_or_method), but expected one of:

(float other)
didn't match because some of the arguments have invalid types: (builtin_function_or_method)
(Variable other)
didn't match because some of the arguments have invalid types: (builtin_function_or_method)

How can I fix this?

Doesn't work without CUDA: RuntimeError: Input type (CUDAFloatTensor) and weight type (CPUFloatTensor) should be the same

danusya@werewolf:~/PyTorch-Multi-Style-Transfer/experiments$ python main.py train --dataset ~/dataset/    --vgg-model-dir caleido_vgg --save-model-dir caleido_model --epochs 2 --cuda 0
/usr/local/lib/python2.7/dist-packages/torchvision/transforms/transforms.py:156: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
Net(
  (gram): GramMatrix(
  )
  (model1): Sequential(
    (0): ConvLayer(
      (reflection_pad): ReflectionPad2d((3, 3, 3, 3))
      (conv2d): Conv2d(3, 64, kernel_size=(7, 7), stride=(1, 1))
    )
    (1): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False)
    (2): ReLU(inplace)
    (3): Bottleneck(
      (residual_layer): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2))
      (conv_block): Sequential(
        (0): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(32, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): ConvLayer(
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(32, 32, kernel_size=(3, 3), stride=(2, 2))
        )
        (6): InstanceNorm2d(32, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(32, 128, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (4): Bottleneck(
      (residual_layer): Conv2d(128, 512, kernel_size=(1, 1), stride=(2, 2))
      (conv_block): Sequential(
        (0): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): ConvLayer(
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2))
        )
        (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
      )
    )
  )
  (ins): Inspiration(N x 512)
  (model): Sequential(
    (0): Sequential(
      (0): ConvLayer(
        (reflection_pad): ReflectionPad2d((3, 3, 3, 3))
        (conv2d): Conv2d(3, 64, kernel_size=(7, 7), stride=(1, 1))
      )
      (1): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False)
      (2): ReLU(inplace)
      (3): Bottleneck(
        (residual_layer): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2))
        (conv_block): Sequential(
          (0): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False)
          (1): ReLU(inplace)
          (2): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1))
          (3): InstanceNorm2d(32, eps=1e-05, momentum=0.1, affine=False)
          (4): ReLU(inplace)
          (5): ConvLayer(
            (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
            (conv2d): Conv2d(32, 32, kernel_size=(3, 3), stride=(2, 2))
          )
          (6): InstanceNorm2d(32, eps=1e-05, momentum=0.1, affine=False)
          (7): ReLU(inplace)
          (8): Conv2d(32, 128, kernel_size=(1, 1), stride=(1, 1))
        )
      )
      (4): Bottleneck(
        (residual_layer): Conv2d(128, 512, kernel_size=(1, 1), stride=(2, 2))
        (conv_block): Sequential(
          (0): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
          (1): ReLU(inplace)
          (2): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1))
          (3): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
          (4): ReLU(inplace)
          (5): ConvLayer(
            (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
            (conv2d): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2))
          )
          (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
          (7): ReLU(inplace)
          (8): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
        )
      )
    )
    (1): Inspiration(N x 512)
    (2): Bottleneck(
      (conv_block): Sequential(
        (0): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): ConvLayer(
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1))
        )
        (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (3): Bottleneck(
      (conv_block): Sequential(
        (0): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): ConvLayer(
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1))
        )
        (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (4): Bottleneck(
      (conv_block): Sequential(
        (0): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): ConvLayer(
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1))
        )
        (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (5): Bottleneck(
      (conv_block): Sequential(
        (0): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): ConvLayer(
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1))
        )
        (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (6): Bottleneck(
      (conv_block): Sequential(
        (0): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): ConvLayer(
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1))
        )
        (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (7): Bottleneck(
      (conv_block): Sequential(
        (0): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): ConvLayer(
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1))
        )
        (6): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (8): UpBottleneck(
      (residual_layer): UpsampleConvLayer(
        (upsample_layer): Upsample(scale_factor=2, mode=nearest)
        (conv2d): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1))
      )
      (conv_block): Sequential(
        (0): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(512, 32, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(32, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): UpsampleConvLayer(
          (upsample_layer): Upsample(scale_factor=2, mode=nearest)
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1))
        )
        (6): InstanceNorm2d(32, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(32, 128, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (9): UpBottleneck(
        (upsample_layer): Upsample(scale_factor=2, mode=nearest)
        (conv2d): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1))
      )
      (conv_block): Sequential(
        (0): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
        (1): ReLU(inplace)
        (2): Conv2d(128, 16, kernel_size=(1, 1), stride=(1, 1))
        (3): InstanceNorm2d(16, eps=1e-05, momentum=0.1, affine=False)
        (4): ReLU(inplace)
        (5): UpsampleConvLayer(
          (upsample_layer): Upsample(scale_factor=2, mode=nearest)
          (reflection_pad): ReflectionPad2d((1, 1, 1, 1))
          (conv2d): Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1))
        )
        (6): InstanceNorm2d(16, eps=1e-05, momentum=0.1, affine=False)
        (7): ReLU(inplace)
        (8): Conv2d(16, 64, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (10): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False)
    (11): ReLU(inplace)
    (12): ConvLayer(
      (reflection_pad): ReflectionPad2d((3, 3, 3, 3))
      (conv2d): Conv2d(64, 3, kernel_size=(7, 7), stride=(1, 1))
    )
  )
)
Traceback (most recent call last):
  File "main.py", line 287, in <module>
    main()
  File "main.py", line 40, in main
    train(args)
  File "main.py", line 159, in train
    style_model.setTarget(style_v)
  File "/home/danusya/PyTorch-Multi-Style-Transfer/experiments/net.py", line 293, in setTarget
    F = self.model1(Xs)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/container.py", line 67, in forward
    input = module(input)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/danusya/PyTorch-Multi-Style-Transfer/experiments/net.py", line 153, in forward
    out = self.conv2d(out)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/conv.py", line 282, in forward
    self.padding, self.dilation, self.groups)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/functional.py", line 90, in conv2d
    return f(input, weight, bias)
RuntimeError: Input type (CUDAFloatTensor) and weight type (CPUFloatTensor) should be the same

vgg16.t7 unhashable type: 'numpy.ndarray

I have put the vgg16.t7 into models folder, it's been detected correctly. However, the following problem happened.

Traceback (most recent call last):
File "main.py", line 295, in
main()
File "main.py", line 41, in main
train(args)
File "main.py", line 135, in train
utils.init_vgg16(args.vgg_model_dir)
File "C:\Users\user\Prepwork\Cap Project\PyTorch-Multi-Style-Transfer\experiments\utils.py", line 100, in init_vgg16
vgglua = load_lua(os.path.join(model_folder, 'vgg16.t7'))
File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 424, in load
return reader.read_obj()
File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 370, in read_obj
obj._obj = self.read_obj()
File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 385, in read_obj
k = self.read_obj()
File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 386, in read_obj
v = self.read_obj()
File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 370, in read_obj
obj._obj = self.read_obj()
File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 387, in read_obj
obj[k] = v
TypeError: unhashable type: 'numpy.ndarray'

It does't work for pytorch-1.0.0 and 1.4.0, and giving the same error, how to deal with it?
thanks !

optim with normal RAM?

Hi,

So I spent around 24 hours so far training a model on my style images, got the results out by using the model on eval and so far they're not great. When I use the optim function with the styles however the results are pretty decent, however I am limited by my VRAM which is 6GB as to what size images I can output. Having a lot more RAM available, I was hoping I could do pretty decently sized images, but it seems that I can only get much larger images with eval. Does eval use normal RAM instead of VRAM?

I will continue training my model so that I can use eval in the future, whether I can do larger images with optim or not, but no idea how much more training is required to make it anywhere near a respectable result.

What sort of overall loss value should I be aiming for? Does the number of style images make a difference to what I should expect?

What is the default value of args.ngf

@zhanghang1989
Thanks for sharing this nice project.

I changed the content of main.py to only have evaluate() method.
def evaluate(content_image_name, style_image_name, output_image_name):
`

content_image = utils.tensor_load_rgbimage(content_image_name, size=1024, keep_asp=True)
content_image = content_image.unsqueeze(0)
style = utils.tensor_load_rgbimage(style_image_name) # size=args.style_size
style = style.unsqueeze(0)
style = utils.preprocess_batch(style)

style_model = Net() # (ngf=args.ngf)
#style_model.load_state_dict(torch.load(args.model))
model_path = 'models/21styles.model'
style_model.load_state_dict(torch.load(model_path))

#if args.cuda: - start
style_model.cuda()
content_image = content_image.cuda()
style = style.cuda()
#if args.cuda - end

style_v = Variable(style, volatile=True)

content_image = Variable(utils.preprocess_batch(content_image), volatile=True)
style_model.setTarget(style_v)

output = style_model(content_image)
utils.tensor_save_bgrimage(output.data[0], output_image_name, 1) #args.cuda=1

Now I am getting below error
While copying the parameter named model1.4.residual_layer.weight, whose dimensions in the model are torch.Size([256, 128, 1, 1]) and whose dimensions in the checkpoint are torch.Size([512, 128, 1, 1]), ...
Traceback (most recent call last):
File "newMain.py", line 297, in
main()
File "newMain.py", line 57, in main
evaluate(content_image_name, style_image_name)
File "newMain.py", line 251, in evaluate
style_model.load_state_dict(torch.load(model_path))
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 360, in load_state_dict
own_state[name].copy_(param)
RuntimeError: inconsistent tensor size, expected tensor [256 x 128 x 1 x 1] and src [512 x 128 x 1 x 1] to have the same number of elements, but got 32768 and 65536 elements respectively at /pytorch/torch/lib/TH/generic/THTensorCopy.c:86

Can you please suggest what I could be missing. Because I have done a very minor change and getting this error when everything is of your orginal code.

running the demo

im getting this error IndexError: too many indices for array

File "main.py", line 290, in <module>
    main()
  File "main.py", line 49, in main
    optimize(args)
  File "main.py", line 102, in optimize
    tbar.set_description(total_loss.data.cpu().numpy()[0])
IndexError: too many indices for array

Link of dataset is broken.

Hi, the link in the download_datast.sh is broken.

It shows ERROR 409: Public access is not permitted on this storage account..
It would be appreciated if you could update the link.
Thanks!

Temporal coherence?

Have you tried some technique for temporal coherence? If not, would you mind if I ask which one would you recommend or would like to try.

Keep up the good work.

Different brush stroke size

In your paper you wrote about the ability to train the model with different sizes of the style images to later get control over the brush stroke size. Did you implement this in either the pytorch or torch implementation?
Greetings and keep up the great work

Training Error, happens with any input style image

Cuda 9
python 2.7

Traceback (most recent call last):
  File "main.py", line 287, in <module>
    main()
  File "main.py", line 40, in main
    train(args)
  File "main.py", line 136, in train
    utils.init_vgg16(args.vgg_model_dir)
  File "/home/ubuntu/train/experiments/utils.py", line 100, in init_vgg16
    vgglua = load_lua(os.path.join(model_folder, 'vgg16.t7'))
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 606, in load_lua
    return reader.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 591, in read
    return self.read_object()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 544, in read_object
    return reader_registry[cls_name](self, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 242, in read_nn_class
    attributes = reader.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 593, in read
    return self.read_table()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 570, in read_table
    v = self.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 593, in read
    return self.read_table()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 570, in read_table
    v = self.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 591, in read
    return self.read_object()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 544, in read_object
    return reader_registry[cls_name](self, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 317, in wrapper
    obj = build_fn(reader, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 242, in read_nn_class
    attributes = reader.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 593, in read
    return self.read_table()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 570, in read_table
    v = self.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 591, in read
    return self.read_object()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 544, in read_object
    return reader_registry[cls_name](self, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 146, in read_tensor
    storage = reader.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 591, in read
    return self.read_object()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 544, in read_object
    return reader_registry[cls_name](self, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 165, in read_storage
    return python_class.from_buffer(reader.f.read(size), 'native')
ValueError: buffer size (53728077) must be a multiple of element size (4)

subtract_imagenet_mean_batch is not working

Hi,

I think the function utils.subtract_imagenet_mean_batch does not work with current code.
You need to add "return batch" in the function and

utils.subtract_imagenet_mean_batch(xc)

should be changed to

xc = utils.subtract_imagenet_mean_batch(xc)

vgg16.t7 unhashable type: 'numpy.ndarray'

It's been a while since the last vgg16 issue i found on this "Issues".

So i download the vgg16.t7 from the paper quoted in this github.
And i run this command "python main.py train --epochs 4 --style-folder images/ownstyles --save-model-dir own_models --cuda 1"
i have put the vgg16.t7 into models folder, it's been detected correctly. However, the following problem happened.

Traceback (most recent call last):
  File "main.py", line 295, in <module>
    main()
  File "main.py", line 41, in main
    train(args)
  File "main.py", line 135, in train
    utils.init_vgg16(args.vgg_model_dir)
  File "C:\Users\user\Prepwork\Cap Project\PyTorch-Multi-Style-Transfer\experiments\utils.py", line 100, in init_vgg16
    vgglua = load_lua(os.path.join(model_folder, 'vgg16.t7'))
  File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 424, in load
    return reader.read_obj()
  File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 370, in read_obj
    obj._obj = self.read_obj()
  File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 385, in read_obj
    k = self.read_obj()
  File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 386, in read_obj
    v = self.read_obj()
  File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 370, in read_obj
    obj._obj = self.read_obj()
  File "C:\Users\user\anaconda3\envs\FTDS\lib\site-packages\torchfile.py", line 387, in read_obj
    obj[k] = v
TypeError: unhashable type: 'numpy.ndarray'

Is there anyway i can fix this? i found in other thread they said replace with another one, but i could not find another one other than from stanford.

Thanks!

how can i test based on different brush size

could you help to check how do we test different brush art style transfer using your code?
thank you!

PyTorch-Multi-Style-Transfer/experiments/utils.py

src_flat_cov_eye = src_norm @ src_norm.t() + Variable(torch.eye(3).cuda())
^
SyntaxError: invalid syntax

@zhanghang1989

？？？？

can not find the code for siamese training

How did you train the siamese net?

Loaded samples aren't normalized and calculated loss is very large?

In the code, loaded samples seem only to substract mean but not to be normalized, which leads to very large loss. Why not normalize the samples?

No way to read and convert to vgg16.weight

No matter how I try, I found that vgg16.t7 has no way to read and convert to vgg16.weight.
Could you help?

Running main.py eval error

Traceback (most recent call last):
File "main.py", line 290, in
main()
File "main.py", line 45, in main
evaluate(args)
File "main.py", line 258, in evaluate
output = utils.color_match(output, style_v)
File "PyTorch-Style-Transfer/experiments/utils.py", line 148, in color_match
matSqrt(src_flat_cov_eye).inverse * src_norm
RuntimeError: mul() received an invalid combination of arguments - got (builtin_function_or_method), but expected one of:

(float other)
didn't match because some of the arguments have invalid types: (builtin_function_or_method)
(Variable other)
didn't match because some of the arguments have invalid types: (builtin_function_or_method)

How do I solve this problem?

The pre-trained model can't access

The connection refused I mean.

Could MSG-Net be used for image enhancement?

Strange result

The first lion was generated with Fast Neural Style Transfer

The second lion was generated with this repo.

It's strange that the picture generated with your code always has some brown area, not only with my picture, but also yours.

I use PyTorch 0.4.0 and the command is

python main.py eval --content-image images/content/cube.png --style-image images/21styles/starry_night.jpg  --model models/21styles.model --content-size 1024

Hope your responce, thank you.

Running the Camera Demo with CPU results in "TypeError: 'torch.FloatTensor' object is not callable"

With Ananoda3 (Python 3.6.5) on Windows, trying to run "python camera_demo.py demo --model models/21styles.model --cuda=0" ended up with the following error.

Traceback (most recent call last):
File "camera_demo.py", line 104, in
main()
File "camera_demo.py", line 101, in main
run_demo(args, mirror=True)
File "camera_demo.py", line 66, in run_demo
simg = style_v.data().numpy()
TypeError: 'torch.FloatTensor' object is not callable

This issue seemed to be fixed with the following change in the line 60 of camera_demo.py
from
simg = style_v.data().numpy()
to
simg = style_v.data[0].numpy()

set default values

Hi,

I try run the camera.py with the arguments discribed in the docs , but fail because inside the code dont have values for args.demo_size and img.copy too
Whats the default values for set these variables?

Thank you

TypeError: 'NoneType' object is not callable

TypeError: 'NoneType' object is not callable
can you check?

Sha or md5 hash for pretrained model

According to BadNets, pretrained models can be poisoned. One way to partially mitigate this is to provide a hash for the downloaded files. It's also just good security practice, especially considering that PyTorch serializes models with Pickle by default, which can execute arbitrary code. Just hoping to make the ML community a bit more secure :)

WHY ARE YOU USING TAB AND SPACE AT SAM TIME????????

training new model

@zhanghang1989
I trained a model with three style images. Now, I see eight .model files.
Can you please tell me which .model file to use OR how to integrate them to single model file.

Thanks
Akash

Super Slow at optim on linux Mint

Have this on a fresh install of linux Mint. I'm running the example, 'python main.py optim --content-image images/content/venice-boat.jpg --style-image images/21styles/candy.jpg' and its taking FOREVER to do anything. I used to have it working at a decent speed on Ubuntu on the same hardware.

When inspecting GPU and CPU usage, I see it start off with minimal GPU usage, and huge CPU usage. it slowly increases GPU usage over time until it has enough and then completes the rest in around the same time as before. As an example, it takes around 8 minutes to figure out that there isn't enough VRAM for the selected image size, whereas previously on my Ubuntu installation that would take a matter of seconds. Any idea why it would take so much longer on Mint? And what I can do to remedy this?

VGG network output

The variable y is the output of the style model and x_c is a copy of the image. Both are run through the VGG network that outputs 4 arrays with 64, 128,512 and 512 maps. To get the content loss, only the second ( features_y[1], features_xc[1].data).

Why was this specific layer selected for the content loss computation, rather that the one(s) with more maps?

RuntimeError: [enforce fail at CPUAllocator.cpp:64] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 34080768 bytes. Error code 12 (Cannot allocate memory)

WHEN RUN ON AWS EC2