The result is so broken,I think a great art style should protect more detail(such as eyes ,nose ,mouth)...So I suggest you can reference this website:
https://www.reddit.com/r/deepdream/comments/4ky9zp/scarlett_johansson_x4/?st=j11ntfv1&sh=fb7f6881

It's talking about how to protect detail and get a great art result;http://imgur.com/a/cH91s

I hold you can try to improve the algorithm for protecting detail ,It will be very beautiful,
It will product a real art like this: http://imgur.com/a/cH91s
Thank you

I'm wondering are there any settings to tune that would give the results a larger styalised look. Ie bigger brush strokes or sketchy lines to make the end result more abstract?

During training what is the result on the model if this flag is set to true?

cmd:option('-reconStyle', false, 'If true, the decoder is also trained to reconstruct style images')

Thanks for your great work! I want to know if there any reason for you to add this layer, is there any experiment result?

what has happened???

Hi,
Thanks for sharing your code publicly.
In your implementation style_loss is summation of mean_square_error of mean and std, this line.

self.mean_loss = self.mean_criterion:forward(self.input_mean, self.target_mean)

But may I ask why you have normalized this value by batch size?

self.mean_loss = self.mean_loss / N -- normalized w.r.t. batch size

Wouldn't MSE criteria take average over N and C?

Hi guys I think I installed unsup dep on mac osx but still getting error:
th test.lua -content input/content/cornell.jpg -style input/style/woman_with_hat_matisse.jpg
/Users/zSUGARBANK/torch/install/bin/luajit: ...rs/zSUGARBANK/torch/install/share/lua/5.1/trepl/init.lua:389: module 'unsup' not found:No LuaRocks module found for unsup

Here's my unsup install log, maybe it's being installed to the wrong place?Any ideas?

Christians-iMac:install zSUGARBANK$ luarocks install /Users/zSUGARBANK/Desktop/unsup-0.1-0.rockspec
Using /Users/zSUGARBANK/Desktop/unsup-0.1-0.rockspec... switching to 'build' mode
Cloning into 'unsup'...
remote: Counting objects: 39, done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 39 (delta 5), reused 14 (delta 1), pack-reused 0
Unpacking objects: 100% (39/39), done.
Checking connectivity... done.
cmake -E make_directory build && cd build && cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="/usr/local/bin/.." -DCMAKE_INSTALL_PREFIX="/usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0" && make

-- The C compiler identification is AppleClang 7.3.0.7030031
-- The CXX compiler identification is AppleClang 7.3.0.7030031
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Torch7 in /usr/local
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/luarocks_unsup-0.1-0-5870/unsup/build
Scanning dependencies of target unsup
[ 50%] Building C object CMakeFiles/unsup.dir/init.c.o
[100%] Linking C shared module libunsup.so
[100%] Built target unsup
cd build && make install
[100%] Built target unsup
Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lib/libunsup.so
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/AutoEncoder.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/ConvPsd.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/Diag.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/FistaL1.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/LinearFistaL1.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/LinearPsd.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/SparseAutoEncoder.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/SpatialConvFistaL1.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/UnsupModule.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/UnsupTrainer.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/init.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/kmeans.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/pca.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/psd.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/topo-kmeans.lua
-- Installing: /usr/local/lib/luarocks/rocks-5.2/unsup/0.1-0/lua/unsup/whitening.lua
Updating manifest for /usr/local/lib/luarocks/rocks-5.2
unsup 0.1-0 is now built and installed in /usr/local (license: BSD)

Thanks so much

Hi would it be possible to implement optical flow with this approach as well?

I was able to implement Your code in pure C# for Windows users: https://github.com/PABCSoft/StyleImitator

I'm tweaking the code to do somewhat of a trivial example: Essentially running any image (in this case a scene, and not a texture), through the AdaIN, and having the style and content be the same image, with the goal of getting an exact reconstruction of the input image (as other style transfer methods can do). However, AdaIN seems to still texturize the output in this procedure, and the output looks a lot like an acuarela-like version of the same image, with most of the fine detail lost. I will try to post an example of this soon.

What is a good suggestion to work around this? I know the training algorithm is not out, but perhaps training the network with non-textures would improve such performance, or just more rounds of training? Thoughts?

Hello,I use VOC2012 as the content image. However,The total number of iterations is 200,000,.I found that the final test effect failed to reach the result of the paper,the choice of the dataset does not matter?Thanks!

I developed an iOS App based on AdaIN. It is free. https://itunes.apple.com/us/app/artx-style-transfer-between-any-two-images/id1246635833?ls=1&mt=8 More info at http://www.artxstyles.com

Where can I get the prototxt file of the original vgg_normalised.caffemodel?

I am wondering that why are you setting std_gradInput_centered = self.std_net:backward(input_view, std_grad) but not std_net:backward(input_centered, std_grad) in StyleINLossModule.lua(line 114) file. I would appreciate it very much if you can give some detailed explanation about how to get the std_gradInput.

I ran the training program and when it complete, there are multiple files under ~/experiments.

dec-160000.t7
history.t7
optimState.t7

Which one is the decoder final results?

Ge

Change at line 266:
local tvLoss = criterion.net:get(2).loss
to:
local tvLoss = criterion.net:get(1).loss

Not sure if I should call the problem as color saturation. In my generated images, I found out there's some pixel chunks, like the black area in the image below.

How should I solve this problem? FYI, the original images are 3232, I resized them to 256256, then train the decoder.

Thanks,

I think it should be strictly above zero for CUDA to be a dependency.

Hi Xunhuang,

We like your paper very much, and reimplement it in TensorFlow here:
https://github.com/tensorlayer/tensorlayer/tree/master/example/adaptive_style_transfer

We hope more people can benefit from this work.

Best wishes,

The noise is so much,Do you have advice for this?

Hi! I am developing applications for Windows in C#. I would like to implement Your algorithm, but for this I need weights of trained neural networks. Could You convert them to JSON or binary format?

Hi,
Loving this implementation, however I'd like to see if it would be possible to achieve "larger style features" if you know what I mean.
Do you think to get bigger more defined brush strokes etc I would retrain with larger style features somehow?

HI
Thanks for your contribution。Can I ask when you will update the training way?Can you tell us the message about the time and memory usage when training;The biggest breakthrough of your reserch is in the transfer efficiency,so I hope see more statistical data;
In a word，you research looks amazing!!!!
Hoping for your update!!!

Can you share the training code?

I don't have GPU so I didn't install cudnn, but when I run the example code:
th test.lua -content input/content/cornell.jpg -style input/style/woman_with_hat_matisse.jpg
It returns errors:

/Users/Bill/torch/install/bin/luajit: /Users/Bill/torch/install/share/lua/5.1/trepl/init.lua:389: module 'cudnn' not found:No LuaRocks module found for cudnn
	no field package.preload['cudnn']
	no file '/Users/Bill/.luarocks/share/lua/5.1/cudnn.lua'
	no file '/Users/Bill/.luarocks/share/lua/5.1/cudnn/init.lua'
	no file '/Users/Bill/torch/install/share/lua/5.1/cudnn.lua'
	no file '/Users/Bill/torch/install/share/lua/5.1/cudnn/init.lua'
	no file './cudnn.lua'
	no file '/Users/Bill/torch/install/share/luajit-2.1.0-beta1/cudnn.lua'
	no file '/usr/local/share/lua/5.1/cudnn.lua'
	no file '/usr/local/share/lua/5.1/cudnn/init.lua'
	no file '/Users/Bill/.luarocks/lib/lua/5.1/cudnn.so'
	no file '/Users/Bill/torch/install/lib/lua/5.1/cudnn.so'
	no file '/Users/Bill/torch/install/lib/cudnn.dylib'
	no file './cudnn.so'
	no file '/usr/local/lib/lua/5.1/cudnn.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
	[C]: in function 'error'
	/Users/Bill/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
	test.lua:41: in main chunk
	[C]: in function 'dofile'
	...Bill/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x0108224eb0

They have train.zip, train_1.zip, .... Which one should I use?

Hi is it possible to run on CPU only?
Is it possible to run on large content images?

th test.lua -content input/content/cornell.jpg -style input/style/woman_with_hat_matisse.jpg

{
content : "input/content/cornell.jpg"
contentDir : ""
outputDir : "output"
styleSize : 512
saveExt : "jpg"
vgg : "models/vgg_normalised.t7"
gpu : 0
saveOriginal : false
crop : false
preserveColor : false
alpha : 1
contentSize : 512
mask : ""
styleDir : ""
decoder : "models/decoder.t7"
style : "input/style/woman_with_hat_matisse.jpg"
styleInterpWeights : ""
}
/home/dxp/torch/install/bin/luajit: /home/dxp/torch/install/share/lua/5.1/trepl/init.lua:389: module 'cudnn' not found:No LuaRocks module found for cudnn
no field package.preload['cudnn']
no file '/root/.luarocks/share/lua/5.1/cudnn.lua'
no file '/root/.luarocks/share/lua/5.1/cudnn/init.lua'

Hello. I want to implement this in Python - Tensorflow. A really great paper.
I read the paper on Adaptive instance normalization, and completely understand the decoder training. However, I am having trouble understanding how the AdaIn part of the algorithm would be written in code. I do not understand Lua and thus do not want to risk misunderstanding the concept.
Can someone explain the AdaIn part of their algorithm to me in pseudocode/english so that I can try to implement it? As far as I am aware, the AdaIn component only takes in the VGG encoder feature maps for the Image and style.

Thank you!

Hello,
I am currently trying to improve AdaIN in a Tensorflow implementation. What do you think can be done to improve the algorithm? What can be done in order to have quality similar to this: https://github.com/lengstrom/fast-style-transfer
I am trying to implement as stated in the paper, histogram losses. I implemented method for histogram matching on 2 images, but i have no idea how to move on.
I need some ideas how to improve the algorithm.

Goal

To make the output image size the same as as the content image size.

Attempt

th test.lua -content mushka_cat0159.jpg -style dark_matter_bw.png -contentSize 0

Error

Out of memory

THCudaCheck FAIL file=/root/torch/extra/cutorch/lib/THC/generic/THCStorage.cu line=66 error=2 : out of memory
/root/torch/install/bin/luajit: /root/torch/install/share/lua/5.1/nn/Container.lua:67:
In 29 module of nn.Sequential:
/root/torch/install/share/lua/5.1/nn/THNN.lua:110: cuda runtime error (2) : out of memory at /root/torch/extra/cutorch/lib/THC/generic/THCStorage.cu:66

[stack traceback omitted]

Setup

Content image
1280x720
25,840 bytes
Download

Style image
512x512
469,165 bytes
Download

GPU
Titan X Pascal

Other attempts

th test.lua -content mushka_cat0159.jpg -style dark_matter_bw.png

Result: output image size: 912x512 (i.e. different than the content image size, 1280x720)

th test.lua -content mushka_cat0159.jpg -style dark_matter_bw.png -contentSize 720

Result: out of memory

Is there any equivalent(converted) caffe model for vgg_normalised.t7 and decoder.t7