dmitryulyanov / texture_nets Goto Github PK

Code for "Texture Networks: Feed-forward Synthesis of Textures and Stylized Images" paper.

License: Apache License 2.0

Lua 99.39% Shell 0.61%

texture-networks torch neural-style style-transfer

texture_nets's Introduction

Texture Networks + Instance normalization: Feed-forward Synthesis of Textures and Stylized Images

In the paper Texture Networks: Feed-forward Synthesis of Textures and Stylized Images we describe a faster way to generate textures and stylize images. It requires learning a feedforward generator with a loss function proposed by Gatys et al.. When the model is trained, a texture sample or stylized image of any size can be generated instantly.

Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis presents a better architectural design for the generator network. By switching batch_norm to Instance Norm we facilitate the learning process resulting in much better quality.

This also implements the stylization part from Perceptual Losses for Real-Time Style Transfer and Super-Resolution.

You can find an oline demo here (thanks to RiseML).

Prerequisites

Torch7 + loadcaffe
cudnn + torch.cudnn (optionally)
display (optionally)

Download VGG-19.

cd data/pretrained && bash download_models.sh && cd ../..

Stylization

Training

Preparing image dataset

You can use an image dataset of any kind. For my experiments I tried Imagenet and MS COCO datasets. The structure of the folders should be the following:

dataset/train
dataset/train/dummy
dataset/val/
dataset/val/dummy

The dummy folders should contain images. The dataloader is based on one used infb.resnet.torch.

Here is a quick example for MSCOCO:

wget http://msvocds.blob.core.windows.net/coco2014/train2014.zip
wget http://msvocds.blob.core.windows.net/coco2014/val2014.zip
unzip train2014.zip
unzip val2014.zip
mkdir -p dataset/train
mkdir -p dataset/val
ln -s `pwd`/val2014 dataset/val/dummy
ln -s `pwd`/train2014 dataset/train/dummy

Training a network

Basic usage:

th train.lua -data <path to any image dataset>  -style_image path/to/img.jpg

These parameters work for me:

th train.lua -data <path to any image dataset> -style_image path/to/img.jpg -style_size 600 -image_size 512 -model johnson -batch_size 4 -learning_rate 1e-2 -style_weight 10 -style_layers relu1_2,relu2_2,relu3_2,relu4_2 -content_layers relu4_2

Check out issues tab, you will find some useful advices there.

To achieve the results from the paper you need to play with -image_size, -style_size, -style_layers, -content_layers, -style_weight, -tv_weight.

Do not hesitate to set -batch_size to one, but remember the larger -batch_size the larger -learning_rate you can use.

Testing

th test.lua -input_image path/to/image.jpg -model_t7 data/checkpoints/model.t7

Play with -image_size here. Raise -cpu flag to use CPU for processing.

You can find a pretrained model here. It is not the model from the paper.

Generating textures

soon

Hardware

The code was tested with 12GB NVIDIA Titan X GPU and Ubuntu 14.04.
You may decrease batch_size, image_size if the model do not fit your GPU memory.
The pretrained models do not need much memory to sample.

Credits

The code is based on Justin Johnson's great code for artistic style.

The work was supported by Yandex and Skoltech.

texture_nets's People

Contributors

Stargazers

Watchers

Forkers

ml-lab wavelets joskid lahwran tienphuocnguyen akkikiki bebee wanjinchang wshenx nguyenductung selimam somaticapi daviehr tybxiaobao geshiming larch18 daidengxin hiyorimi zomeelee wannasmile mvpduncan alanyannick weixiaee gouxiayibu michalbusta alululululululu arnabgho veterun rhythm92 baiyancheng20 panyang eriche2016 linzhineng empia chenglongchen szhou0731 codeaudit caomw chagge dawume benjamesbabala jacobo3d dreadlord1984 xpeng zuraguerra liuguoyou colastaralex bearprada andyxiaoprince mstrazds jinzhiyong123 dnuffer xiangliu886 kaca0083 yzlryo sttu clcarwin vicident lijian8 sarikayamehmet criminalking alihassan1 markzhong88 mungobungo aravindhm gouravd ml-ai-nlp-ir sdujump nandexsp nanyomy sdorminey ezioruan zhsso ilovecv anotherun johnsmall albertxiebnu dwangarc lab-x royal-feng mphielipp wang4959520 walkoncross whusky leizi007 zmxheart bemoregt bradparks dkorolev chenbangfeng jnl64bit aboody2009 chunniunai220ml crowdsdom kuixu vyraun alexguo zhouqingping zhixinshu johndpope

texture_nets's Issues

Run Test.lua and Train.lua, lots of errors

ImageSize

Hi, I found that your stylization_process couldn't adaptive to different resolution,just using the resolution 800*800,etc. I fixed it.

modified the models/johnson.lua's network structure and got wrong results

I tried to modified the models/johnson.lua's network structure, aimed to reduce running memory and shorten running time, just as follows:

model:add(pad(4, 4, 4, 4))
model:add(backend.SpatialConvolution(3, 32, 9, 9, 2, 2, 0, 0)) --src_stride=1
model:add(normalization(32))
model:add(nn.ReLU(true))
..........
..........
model:add(nn.SpatialFullConvolution(32, 3, 3, 3, 2, 2, 1, 1, 1, 1)) --source is SpatialConvolution
model:add(normalization(3))
model:add(nn.ReLU(true))

return model:add(nn.TVLoss(params.tv_weight))

and training dataSet is COCO, image_size=512, style_size=512, content_weight=10, style_weight=10, tv_weight=0.001, I also tried others content_weight and style_weight, it seemed it had no significant difference, here is my result image by tubingen_kandinsky.jpg

Error plots for Training

Can you share the error plots for training? How many times does the error get reduced when compared to initial error?

Instance normalization implemented?

Hi Dmitry! has the instance normalization layer been implemented already? I haven't found reference in the code. Maybe you can refer me to the file/function that contains this addition. Thanks!

Faster on 84GHz / 32G Mem than 322GHz / 64G Mem ?

I've tested these on a computer with AMD FX(tm)-8350 Eight-Core Processor, with 32G RAM installed, with test.lua, I can get a stylized.jpg in 6 seconds. After reading this post #41
I think it's possible to generate high resolution images if I have larger RAM.
I got a VM running on XenServer, with 32 CPU core(Intel(R) Xeon(R) CPU E7- 4820 @ 2.00GHz) and 64G RAM, but when I ran test.lua with regular params, it was extreamly slow than the former computer.
It costs over 15 minutes to generate one single image. What's wrong with that?
I noticed the script runs on only single one CPU core, is that normal?

This is what I used:
th test.lua -input_image images/forbidden_city.jpg -model_t7 data/checkpoints/model.t7 -cpu

Any body have experience on this?

Thanks.

More parameter about the trainning model

Hi,
I have used the model by your download link, the result look nice. But can you provide more input how you train this model, for example the content and style weight and also is it possible to output original ratio size photo rather than complete square output? Also if you can provide more models for test would be appreciated! Anyway, awesome job on the feed forward method.

Thanks

Training the model failed

Hi,
I tried to train my model based on the imagenet validation set which contains 50k images. At the beginning, i.e. the iteration step is smaller than some small numbers like 8000, I can get reasonable test result using the trained model. But as the training going on, I get a black image, where is output of the pixel value is NaN! All the parameters used for training are unchanged, and the training image is the " cezanne.jpg" which is included in the branch of texture_nets_v1.
Shoud I change the learning late? Could you please give me some advice about this problem?
Thanks!

How to use gpu mode

well, i run the pretrain model code , and i encounter the problem:attempt to index field 'gradInput' (a nil value).And i think i use the gpu, and what's wrong?

About the "border effect" in result

Hi, i'm training a model with style "van gogh's starry night" and got some result as follows. there are some weird border like effects in result image no matter how i tune the weight or tvloss parameters. i also tried different image size or style size and even with cropped style image, the results' quality is slightly different but none of them the border effect is disappear.

different content image with the same effect:

i use these parameters: -model johnson -image_size 512 -style_size 512 -content_weight 10 -style_weight 500 -learning_rate 0.001 -normalize_gradients true -tv_weight 0.000085

i checked the result after training 1000, 5000,10000 and 40000 iters, things didn't change.

has any body meet the same problem? any suggestion is appreciated, thanks!

train model seems not OK?

I use

th train.lua -data -style_image path/to/img.jpg

it gen
model_1000.t7
model_2000.t7
model_3000.t7
......
model_49000.t7
model_50000.t7

but train model_50000.t7 to gen new picture seems not ok, is any wrong from my operate?

th test.lua -input_image tubingen.jpg -model_t7 data/checkpoints/rio.jpg/model_50000.t7

rio.zip

thanks

train models without GPU

is it possible to train models without GPU?
always get error: module 'cutorch' not found

Is it possible to scale style when generating image after training(using test.lua)

Hi, i find i can't tune the style scale in result using test.lua, also i can't find any option about it or is it possible in theory to scale style after the training procedure?

recently i trained a style, and created the following results on a photo with size 668 x 448 and with its original size 4272 × 2848:

668 x 448

4272 x 2848

i can understand both of the results are right, and the style is applied correctly. but if i can tune the style scale on large size image as artist using large brush, i think it will be better to user to see consistency between thumbnail(he get this quickly) and the final size result(he get this after a long time).

sorry i don't know if this is mentioned on paper or there is any progress in recently research?

Train results. Model.state ?

Hi, for what is model.state? thanks

getting error while training

while training the data this error is coming.

threads/threads.lua:183: [thread 1 callback] bad argument #2 to '?' (dimension 1 out of range of 0D tensor at /home/xyz/torch/pkg/torch/generic/Tensor.c:19) stack traceback: [C]: at 0x7ff3c8ad2830 [C]: in function 'xpcall' .../xyz/torch/install/share/lua/5.1/threads/threads.lua:234: in function 'callback' ...t6/xyz/torch/install/share/lua/5.1/threads/queue.lua:65: in function <...t6/xyz/torch/install/share/lua/5.1/threads/queue.lua:41> [C]: in function 'pcall' ...t6/xyz/torch/install/share/lua/5.1/threads/queue.lua:40: in function 'dojob' [string " local Queue = require 'threads.queue'..."]:13: in main chunk stack traceback: [C]: in function 'error' .../xyz/torch/install/share/lua/5.1/threads/threads.lua:183: in function 'dojob' .../xyz/torch/install/share/lua/5.1/threads/threads.lua:264: in function 'synchronize' .../xyz/torch/install/share/lua/5.1/threads/threads.lua:142: in function 'specific' .../xyz/torch/install/share/lua/5.1/threads/threads.lua:125: in function 'Threads' ./dataloader.lua:52: in function '__init' ...xyza/torch/install/share/lua/5.1/torch/init.lua:91: in function <...xyz/torch/install/share/lua/5.1/torch/init.lua:87> [C]: in function 'DataLoader' ./dataloader.lua:28: in function 'create' train.lua:79: in main chunk [C]: in function 'dofile' ...xyz/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670

How to Set Up Training Data

I'm trying to train a style image using the MS COCO dataset but am having trouble setting up the directories in the proper way. I tried to follow your instructions in the README and have the following structure:

image-training/
image-training/train
image-training/train/dummy (images are in this folder)
image-training/val
image-training/val/dummy (images are in this folder)

When I run this command th train.lua -data image-training/ -style_image data/textures/cezanne.jpg, I get the following error:

/root/torch/install/bin/luajit: /root/texture_nets/datasets/style-gen.lua:67: class not found: val

Do you know if the structure of my image-training folder is incorrect?

Typos in your paper?

Hi, I noticed a place( in part Network architecure in Section 3.2 Generator network for texture synthesis ) in your paper on texture nets where you mentioned that you use different scale of noise tensors which are 1/2M, 1/4M, ....of the image size repectively. However, in your code on texture nets, you do use noise tensor of size M x M, which is the same as the input image size. Does this means that in your paper, variable i should be choosen to be i = 0， 1, ..., K-1, other than i starting from index 1 up to K

is it possible to resume training progress from checkpoint obtained before(snapshot)

Hi guys, i meet this certain situation, i confirm an amazing effect on some checkpoint but the training progress is stopped before. so i think if i can continue the training instead of discarding those intermediate result(for example have been trained 8K+ iters) and starting an new one(this will consume more unnecessary time), this will be very helpful to save our time.

or if i missed something in train.lua?

thanks in advance.

Problem in model/pyramid.lua

Original code:

        -- Batch norm before merging   -- line 40
        seq:add(bn(conv_num))          -- line 41
        cur_temp:add(bn(conv_num))     -- line 42

Maybe the code below could be better:

        -- Batch norm before merging       -- line 40
        seq:add(bn(conv_num))              -- line 41
        cur_temp:add(bn(conv_num*(i-1)))   -- line 42

Not running in evaluation mode during testing/generation intentional?

It seems to me that the network is not being run in evaluation mode (and therefore use the population mean/variance for the batch/instance normalization layers) when running feed forward/generating styled images. Is this intentional? I read your paper and wasn't sure if

Differently from batch normalization, furthermore, the instance normalization layer is applied at test time as well

meant to calculate the means/variances during test time as well/not use population statistics.

All brushes same size

Hi! It looks like there is a problem where all the "brush strokes" in a style are all the same size. You can really see it here:

This was the style image I used:

40,000 iterations of the microsoft coco dataset. styleweight of 3, stylesize of 512, image size of 512, learning set to .001, batch of 1. I got the same results with different style weights and learning sizes. Unfortunately this looks like a big limitation :(

Symbol lookup error

Just decided to have a bit of fun with stylized images, but I got the symbol lookup error. I have installed all the needed dependencies through the script several times, but in vain.

Here is what I got:

Successfully loaded data/pretrained/VGG_ILSVRC_19_layers.caffemodel
conv1_1: 64 3 3 3
conv1_2: 64 64 3 3
conv2_1: 128 64 3 3
conv2_2: 128 128 3 3
conv3_1: 256 128 3 3
conv3_2: 256 256 3 3
conv3_3: 256 256 3 3
conv3_4: 256 256 3 3
conv4_1: 512 256 3 3
conv4_2: 512 512 3 3
conv4_3: 512 512 3 3
conv4_4: 512 512 3 3
conv5_1: 512 512 3 3
conv5_2: 512 512 3 3
conv5_3: 512 512 3 3
conv5_4: 512 512 3 3
fc6: 1 1 25088 4096
fc7: 1 1 4096 4096
fc8: 1 1 4096 1000
conv1_1 : padding set to 0  
Setting up texture layer    2   :   relu1_1 
/usr/bin/luajit: symbol lookup error: /usr/lib/lua/5.1/libTHCUNN.so: undefined symbol: THC_overlappingIndices

Thank you for your help.

Parameter used in Figure 3?

Figure 3 in "Texture Networks: Feed-forward Synthesis of Texture and Stylized Images" is fantastic. For style of Picasso's painting (first row of figure 3), I trained the model with default parameter. But the result with style of Picasso's painting is bad.
Can you share the parameter setting for style of Picasso's painting ,or the trained model?
Thank you very much!

Does instance normalisation support in current version of texture_nets?

Hi,
Does the current version texture_nets support instance normalisation? I could not find function like that in the project, instead, a BatchNormalisation found in the utils.lua.

Best

Stylization train error...

Command:
th stylization_train.lua -style_image data/textures/style.jpg -train_hdf5 data/256.hdf5 -noise_depth 3 -model_name pyramid2 -normalize_gradients true -train_images_path data/256.hdf5 -content_weight 0.8 -backend cudnn

Error:
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:537] Reading dangerously large protocol message. If the message turns out to be larger than 1073741824 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 574671192
Successfully loaded data/pretrained/VGG_ILSVRC_19_layers.caffemodel
conv1_1: 64 3 3 3
conv1_2: 64 64 3 3
conv2_1: 128 64 3 3
conv2_2: 128 128 3 3
conv3_1: 256 128 3 3
conv3_2: 256 256 3 3
conv3_3: 256 256 3 3
conv3_4: 256 256 3 3
conv4_1: 512 256 3 3
conv4_2: 512 512 3 3
conv4_3: 512 512 3 3
conv4_4: 512 512 3 3
conv5_1: 512 512 3 3
conv5_2: 512 512 3 3
conv5_3: 512 512 3 3
conv5_4: 512 512 3 3
fc6: 1 1 25088 4096
fc7: 1 1 4096 4096
fc8: 1 1 4096 1000
Setting up texture layer 2 : relu1_1
Setting up texture layer 7 : relu2_1
Setting up texture layer 12 : relu3_1
Setting up texture layer 21 : relu4_1
Setting up content layer 23 : relu4_2
Setting up texture layer 30 : relu5_1
WARNING: Skipping content loss
Optimize
/home/ai2/torch/install/bin/luajit: stylization_train.lua:110: attempt to concatenate a nil value
stack traceback:
stylization_train.lua:110: in function 'get_input_train'
stylization_train.lua:141: in function 'opfunc'
/home/ai2/torch/install/share/lua/5.1/optim/adam.lua:33: in function 'optim_method'
stylization_train.lua:181: in main chunk
[C]: in function 'dofile'
.../ai2/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00405d50

Any idea ???

train model show black photo after 10000.t7

Hi I use this setting:
th train.lua -data datasets/ -style_image ../new_style/cosmo512.jpg -tv_weight 0.001

I can see the output image between model_1000.t7 to model_10000.t7. But after that, all the test image show black photo, is there something wrong with the training data?

CPU mode

hello! any chance you can help me trying to make it work on CPU only mode? I read about the Batchnormalization error due to your mistake, can you give details?
thanks!

What parameters used in paper's fig.4?

The result in paper's fig.4 was excellent. But I trained the model by the code with the default parameter, the result was not so good.

Error: bad argument #3 to '?' (number expected, got nil)

Command:
th stylization_process.lua -model supplementary/stylization_models/mosaic_style.t7 -input_image data/textures/tiger.jpg -noise_depth 3

/home/ai2/torch/install/bin/luajit: /home/ai2/torch/install/share/lua/5.1/nn/Container.lua:67:
In 1 module of nn.Sequential:
In 1 module of nn.Concat:
In 1 module of nn.Sequential:
In 1 module of nn.Concat:
In 1 module of nn.Sequential:
In 1 module of nn.Concat:
In 1 module of nn.Sequential:
In 1 module of nn.Concat:
In 2 module of nn.Sequential:
bad argument #3 to '?' (number expected, got nil)
stack traceback:
[C]: at 0x7fd61fcb31a0
[C]: in function '__newindex'
./src/utils.lua:148: in function <./src/utils.lua:144>
[C]: in function 'xpcall'
/home/ai2/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
/home/ai2/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function </home/ai2/torch/install/share/lua/5.1/nn/Sequential.lua:41>
[C]: in function 'xpcall'
/home/ai2/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
/home/ai2/torch/install/share/lua/5.1/nn/Concat.lua:12: in function </home/ai2/torch/install/share/lua/5.1/nn/Concat.lua:9>
[C]: in function 'xpcall'
...
[C]: in function 'xpcall'
/home/ai2/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
/home/ai2/torch/install/share/lua/5.1/nn/Concat.lua:12: in function </home/ai2/torch/install/share/lua/5.1/nn/Concat.lua:9>
[C]: in function 'xpcall'
/home/ai2/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
/home/ai2/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward'
stylization_process.lua:38: in main chunk
[C]: in function 'dofile'
.../ai2/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00405d50

WARNING: If you see a stack trace below, it doesn't point to the place where this error occured. Please use only the one above.
stack traceback:
[C]: in function 'error'
/home/ai2/torch/install/share/lua/5.1/nn/Container.lua:67: in function 'rethrowErrors'
/home/ai2/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward'
stylization_process.lua:38: in main chunk
[C]: in function 'dofile'
.../ai2/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00405d50

Could you please provide a pretrained model for testing?

Hi, It's great to see the code updating. But I cannot find a pretrained model in this version. It will be very nice if you can upload a pretrained model by which I can have a test quickly!

Thank you~

Blank Images using Pyramid model

I am attempting to train a model using the default pyramid model with this command (also tried using all defaults):

th train.lua -style_weight 5   -tv_weight 0.000085  -style_size 512 -model pyramid  -image_size 512 -num_iterations 50000  -batch_size 1 -data ../coco2014 -style_image ../candy_512.jpg

When I try to generate an image with it, I am getting black output, even though the input image's dimension is a multiple of 32:

th test.lua  -image_size 1024  -input_image ../Input_square1568.jpg  -model_t7 pyramid/model_20000.t7  -save_path stylized.jpg

The issue occurs if I don't use the image_size flag as well. The weird thing is, it seemed to work a couple times, but now all I get is black output. Using johnson's model does not create the issue, but skip_unpool model also creates blank output.

I have tried creating smaller images to 256, 512 etc. but they are all blank.

I am just trying to figure out how to train the model to achieve results similar to the default model. As others have posted, it doesn't seem possible with the current code. It seems like others have experienced the black output as well. Please advise!

...torch/install/share/lua/5.1/cudnn/SpatialConvolution.lua:108: input has to contain: 16 feature maps, but received input of size: 1 x 19 x 29 x 29

When I ran the code:th stylization_process.lua -input_image data/readme_pics/kitty.jpg -model data/out/model.t7 -noise_depth 16 -save_path stylized.png.It just told me:...torch/install/share/lua/5.1/cudnn/SpatialConvolution.lua:108: input has to contain: 16 feature maps, but received input of size: 1 x 19 x 29 x 29.
The model.t7 is the one that produced by:th texture_train.lua -texture data/textures/red-peppers256.o.jpg -model_name pyramid -backend cudnn -num_iterations 1500 -vgg_no_pad true -normalize_gradients true -batch_size 15.
I don't know where is wrong.Could you give a solution?Thanks a lot.
Looking forward to your reply.

Error save model

/home/ml/torch/install/bin/luajit: /home/ml/torch/install/share/lua/5.1/torch/File.lua:141: Unwritable object at ..modules.2.<?>.biasDesc
stack traceback:
[C]: in function 'error'
/home/ml/torch/install/share/lua/5.1/torch/File.lua:141: in function 'writeObject'
/home/ml/torch/install/share/lua/5.1/torch/File.lua:235: in function 'writeObject'
/home/ml/torch/install/share/lua/5.1/nn/Module.lua:154: in function 'write'
/home/ml/torch/install/share/lua/5.1/torch/File.lua:210: in function 'writeObject'
/home/ml/torch/install/share/lua/5.1/torch/File.lua:235: in function 'writeObject'
/home/ml/torch/install/share/lua/5.1/torch/File.lua:235: in function 'writeObject'
/home/ml/torch/install/share/lua/5.1/nn/Module.lua:154: in function 'write'
/home/ml/torch/install/share/lua/5.1/torch/File.lua:210: in function 'writeObject'
/home/ml/torch/install/share/lua/5.1/torch/File.lua:388: in function 'save'
train.lua:203: in main chunk
[C]: in function 'dofile'
...e/ml/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00406670

Out of memory

Hi. out of memory at /home/whr/torch-cl/extra/cutorch/lib/THC/generic/THCStorage.cu:40 when i run the "th texture_train.lua -texture data/textures/red-peppers256.o.jpg -model_name pyramid -backend cudnn -num_iterations 1500 -vgg_no_pad true -normalize_gradients true". My GPU is Nvidia Black Titan X. What's wrong? Thx.

How to train data

I am training on MS_COCO dataset. i am running this command

th train.lua -data /train/dummy -style_image data/textures/cezanne.jpg

but i am getting error
/usr/local/bin/luajit: /usr/local/share/lua/5.1/trepl/init.lua:383: module 'display' not found:No LuaRocks module found for display no field package.preload['display'] no file '/home/bfomitchev/.luarocks/share/lua/5.1/display.lua' no file '/home/bfomitchev/.luarocks/share/lua/5.1/display/init.lua' no file '/usr/local/share/lua/5.1/display.lua' no file '/usr/local/share/lua/5.1/display/init.lua' no file './display.lua' no file '/usr/local/share/luajit-2.1.0-beta1/display.lua' no file '/home/guest6/.luarocks/share/lua/5.1/display.lua' no file '/home/guest6/.luarocks/share/lua/5.1/display/init.lua' no file '/home/bfomitchev/.luarocks/lib/lua/5.1/display.so' no file '/usr/local/lib/lua/5.1/display.so' no file './display.so' no file '/usr/local/lib/lua/5.1/loadall.so' no file '/home/guest6/.luarocks/lib/lua/5.1/display.so' stack traceback: [C]: in function 'error' /usr/local/share/lua/5.1/trepl/init.lua:383: in function 'require' train.lua:10: in main chunk [C]: in function 'dofile' /usr/local/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670

Thanks

Is display used or needed

I am trying to figure out how to make use of display. Is there a default IP/Port used to connect via the browser to view images at every 50th iteration?

I am not seeing ip/port initialisation in the code so I assume there must be some default values like 127.0.0.1:80... but I don't get anything there.

Based on the actual code display need to be installed for train to work... so it appear to be a hard requirement.

Bad argument #1 to 'copy' (sizes do not match) at model:forward(input:cuda())

Hi I have this error at line 33 of test.lua :

torch/install/share/lua/5.1/nn/Concat.lua:25: bad argument #1 to 'copy' (sizes do not match at /tmp/luarocks_cutorch-scm-1-9130/cutorch/lib/THC/THCTensorCopy.cu:30)
stack traceback:
    [C]: in function 'copy'

at the line 33 of test.lua when doing model:forward(input:cuda())
It's doing that with my own model I have pretrained with train.lua, it's not doing that with the pretrained model downloaded from the readme...

How do I use my own pretrained styles ?

Thanks in advance :) .

Any advice about to get a plausible model of my own style input

hi @DmitryUlyanov

Recently, I have used your code to train my own style. I finish the training process, however, it it very difficult to get a plausible result based my own image style. There are many hyper parameters may affect the result. So any advice to get a stable model?

test.lua "table index is nil"

root@ubuntu:/home/fengbin/texture2# th test.lua -input_image 11.jpg -model_t7 model.t7 -cpu
/usr/local/bin/luajit: /usr/local/share/lua/5.1/torch/File.lua:361: table index is nil
stack traceback:
/usr/local/share/lua/5.1/torch/File.lua:361: in function 'readObject'
/usr/local/share/lua/5.1/nn/Module.lua:154: in function 'read'
/usr/local/share/lua/5.1/torch/File.lua:342: in function 'readObject'
/usr/local/share/lua/5.1/torch/File.lua:391: in function 'load'
test.lua:17: in main chunk
[C]: in function 'dofile'
/usr/local/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x0804d6d0

Resolution Limitation

I see that the pretrained model you provided runs at higher resolutions compared to the models I trained for a few style images. What might be the reason for this?Did you use VGG19 or normalised vgg?

Problem in GenNoise Class

I've run the command:

th stylization_process.lua -model supplementary/stylization_models/starry_style.t7 -input_image data/readme_pics/kitty.jpg -noise_depth 3

I get an error which corresponds with the GenNoise class updateOuput:

/home/siavash/torch/install/bin/luajit: bad argument #3 to '?' (number expected, got nil) stack traceback: [C]: at 0x7f10f8684f30 [C]: in function '__newindex' ./src/utils.lua:138: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Concat.lua:12: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Concat.lua:12: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Concat.lua:12: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Concat.lua:12: in function 'updateOutput' /home/siavash/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward' stylization_process.lua:37: in main chunk [C]: in function 'dofile' ...vash/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00405d70

Correct params for train

Hi,

what are the correct params for train? I'm trying to train a model but the results are wrong :(

image.load() error

I'm getting the below error when I try to run test.lua on any image that wasn't included in the repository. Even if I download an image from the repository (i.e. https://raw.githubusercontent.com/DmitryUlyanov/texture_nets/master/data/readme_pics/karya.jpg) I get the same error. Has anyone else run into this?

/root/torch/install/bin/luajit: /root/torch/install/share/lua/5.1/image/init.lua:220: Not a JPEG file: starts with 0x2d 0x2d
stack traceback:
    [C]: in function 'load'
    /root/torch/install/share/lua/5.1/image/init.lua:220: in function 'loader'
    /root/torch/install/share/lua/5.1/image/init.lua:368: in function 'load'
    test.lua:23: in main chunk
    [C]: in function 'dofile'
    /root/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
    [C]: at 0x00406670

Similarly, for PNGs I get

/root/torch/install/bin/luajit: /root/torch/install/share/lua/5.1/image/init.lua:156: [read_png_file] File data/readme_pics/house.png is not recognized as a PNG file
stack traceback:
    [C]: in function 'load'
    /root/torch/install/share/lua/5.1/image/init.lua:156: in function 'loader'
    /root/torch/install/share/lua/5.1/image/init.lua:368: in function 'load'
    test.lua:23: in main chunk
    [C]: in function 'dofile'
    /root/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
    [C]: at 0x00406670

Using Multi-GPUs

The training process usually need one hour. Is it possible to use multi-gpus for training ?

Stylization train params

Exemple:
th stylization_train.lua -style_image data/textures/cezanne.jpg -train_hdf5 <path/to/generated/hdf5> -noise_depth 3 -model_name pyramid -normalize_gradients true -train_images_path <path/to/ILSVRC2012> -content_weight 0.8

Can you please explain every param:
-style_image - path to style image (.jpg, .png(??))
-train_hdf5 - path to generated hdf5 file, generated by this command (th scripts/extract4_2.lua -images_path <path/ILSVRC2012>) ???
-train_images_path - ???

Unable to get good results

Ive been trying to train the network for the Van Gogh starry night style by modifying the style_weight and tv_weight (0.001). I am not able to get good results.

Any suggestions on how to improve this? Has anyone tried using Van Gogh starry night? How can i reduce the blocky appearance?

error: `module 'src/utils.lua' not found`

I tried
th texture_train.lua -texture data/textures/red-peppers256.o.jpg -model_name pyramid -backend nn -num_iterations 1500 -vgg_no_pad -normalize_gradients
but there's an error message: module 'src/utils.lua' not found:No LuaRocks module found for src/utils.lua

I also used th in the root directory of texture_net, and typed require 'src/utils.lua', the error message was the same.

I'm very new to Torch, and I almost installed all the packages. It seems like a path setting issue, but I have no idea how to fix it. Could anyone give me some hint?
Thanks!

can test.lua support CPU mode?

Hi, since my GPU is not much good for full image size in test method, and i want to check some .t7 models generated from other machine. i think if test can support CPU(the speed is not too slow i think anyway), it will be more helpful.

Training model failed. class not found: val

I have this directory structure:
DmitryCode
--- train
--- val
--- texture_nets

And when I launch: "th train.lua -data ../ -style_image style.jpg"
I get this error :(
"
=> Generating list of images
| finding all validation images
find: ‘standard output’: Tubería rota
find: write error
/home/pixie/pixify/torch/install/bin/luajit: /home/pixie/pixify/trainlua/datasets/style-gen.lua:67: class not found: val
stack traceback:
[C]: in function 'assert'
/home/pixie/pixify/trainlua/datasets/style-gen.lua:67: in function 'findImages'
/home/pixie/pixify/trainlua/datasets/style-gen.lua:102: in function 'exec'
./datasets/init.lua:28: in function 'create'
./dataloader.lua:24: in function 'create'
train.lua:79: in main chunk
[C]: in function 'dofile'
...xify/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x004064d0
"

Somebody knows how could I fix this? Thanks

choice of GPU

Hi! I didn't manage to find in your code an option to choose the number of graphics card. If there is such an option, how to use it?
Thank you in advance.
Sincerely