Light

CUDA memory issues about chainer-fast-neuralstyle HOT 11 CLOSED

yusuketomoto commented on July 16, 2024

CUDA memory issues

from chainer-fast-neuralstyle.

Comments (11)

ttoinou commented on July 16, 2024 1

Well it's slow for the GPU but it might be because of the model loading not the generation of the image. By stats I was talking about the memory :) .
Resize the image bigger and bigger to see the memory consumption and when it breaks.
( if it's written 1000MB and has ran out of memory doesn't mean that he was only trying to allocate 1000 MB but much more and you can't see it in the stats. I think 1000 MB is just the memory he succeeded in allocating )

from chainer-fast-neuralstyle.

yusuketomoto commented on July 16, 2024

Yes. It's hard to train on MacbookPro with batches due to VRAM capacity.
I think buying GPU is the best choice...
There is an another option, In this code, input images are resized to (256,256), you can change image size smaller. But it could also spoil the quality.

If you installed chainer after CuDNN installation, CuDNN should be enabled. Otherwise, CuDNN isn't used.

from chainer-fast-neuralstyle.

genekogan commented on July 16, 2024

or maybe using a smaller network than VGG_16 (NIN ImageNet?) would help? thanks for the tip, will try to find a GPU to use.

from chainer-fast-neuralstyle.

valentinvieriu commented on July 16, 2024

@genekogan Try to see if you have a scaled resolution on your mac, and if yes change the resolution to retina and try again. In my case this solved the issue.

from chainer-fast-neuralstyle.

wadkar commented on July 16, 2024

I have CuDNN installed and enabled (paths set correctly) but I still get the out of memory exception.

I am trying to generate output for an image file that is 2.2mb in size (12px image captured from a smartphone). Any hints in this direction?

from chainer-fast-neuralstyle.

wadkar commented on July 16, 2024

@yusuketomoto thanks, using CPU for full resolution and convert -resize 30% ... worked for GPU

from chainer-fast-neuralstyle.

fortunto2 commented on July 16, 2024

Hi, i install Chainer with CUDA on AWS GPU x2 instance
without CuDNN

~/chainer-fast-neuralstyle$ python generate.py sample_images/tubingen.jpg -m models/seurat.model -g 0 -o sample_images/output.jpg
/usr/local/lib/python2.7/dist-packages/chainer/cuda.py:87: UserWarning: cuDNN is not enabled.
Please reinstall chainer after you install cudnn
(see https://github.com/pfnet/chainer#installation).
  'cuDNN is not enabled.\n'
Traceback (most recent call last):
  File "generate.py", line 30, in <module>
    y = model(x)
  File "/home/ubuntu/chainer-fast-neuralstyle/net.py", line 61, in __call__
    h = self.r4(h, test=test)
  File "/home/ubuntu/chainer-fast-neuralstyle/net.py", line 20, in __call__
    h = F.relu(self.b1(self.c1(x), test=test))
  File "/usr/local/lib/python2.7/dist-packages/chainer/links/connection/convolution_2d.py", line 82, in __call__
    x, self.W, self.b, self.stride, self.pad, self.use_cudnn)
  File "/usr/local/lib/python2.7/dist-packages/chainer/functions/connection/convolution_2d.py", line 316, in convolution_2d
    return func(x, W, b)
  File "/usr/local/lib/python2.7/dist-packages/chainer/function.py", line 130, in __call__
    outputs = self.forward(in_data)
  File "/usr/local/lib/python2.7/dist-packages/chainer/function.py", line 234, in forward
    return self.forward_gpu(inputs)
  File "/usr/local/lib/python2.7/dist-packages/chainer/functions/connection/convolution_2d.py", line 131, in forward_gpu
    cover_all=self.cover_all)
  File "/usr/local/lib/python2.7/dist-packages/chainer/utils/conv.py", line 45, in im2col_gpu
    col = cuda.cupy.empty((n, c, kh, kw, out_h, out_w), dtype=img.dtype)
  File "/usr/local/lib/python2.7/dist-packages/cupy/creation/basic.py", line 20, in empty
    return cupy.ndarray(shape, dtype=dtype)
  File "cupy/core/core.pyx", line 87, in cupy.core.core.ndarray.__init__ (cupy/core/core.cpp:5019)
  File "cupy/cuda/memory.pyx", line 275, in cupy.cuda.memory.alloc (cupy/cuda/memory.cpp:5517)
  File "cupy/cuda/memory.pyx", line 414, in cupy.cuda.memory.MemoryPool.malloc (cupy/cuda/memory.cpp:8078)
  File "cupy/cuda/memory.pyx", line 430, in cupy.cuda.memory.MemoryPool.malloc (cupy/cuda/memory.cpp:8004)
  File "cupy/cuda/memory.pyx", line 337, in cupy.cuda.memory.SingleDeviceMemoryPool.malloc (cupy/cuda/memory.cpp:6972)
  File "cupy/cuda/memory.pyx", line 357, in cupy.cuda.memory.SingleDeviceMemoryPool.malloc (cupy/cuda/memory.cpp:6799)
  File "cupy/cuda/memory.pyx", line 255, in cupy.cuda.memory._malloc (cupy/cuda/memory.cpp:5459)
  File "cupy/cuda/memory.pyx", line 256, in cupy.cuda.memory._malloc (cupy/cuda/memory.cpp:5380)
  File "cupy/cuda/memory.pyx", line 31, in cupy.cuda.memory.Memory.__init__ (cupy/cuda/memory.cpp:1542)
  File "cupy/cuda/runtime.pyx", line 181, in cupy.cuda.runtime.malloc (cupy/cuda/runtime.cpp:3065)
  File "cupy/cuda/runtime.pyx", line 111, in cupy.cuda.runtime.check_status (cupy/cuda/runtime.cpp:1980)
cupy.cuda.runtime.CUDARuntimeError: cudaErrorMemoryAllocation: out of memory

from chainer-fast-neuralstyle.

ttoinou commented on July 16, 2024

@fortunto2 : The error might be that... you don't have enough memory. I don't understand your point.

from chainer-fast-neuralstyle.

fortunto2 commented on July 16, 2024

max value 1015MiB
in interval 1sec
nvidia-smi -l 1

from chainer-fast-neuralstyle.

ttoinou commented on July 16, 2024

Could you try with smaller image sizes in order to see the last stats before it crashes ?

from chainer-fast-neuralstyle.

fortunto2 commented on July 16, 2024

yes work with small size
gpu 0.48 sec
cpu 0.59 sec
it's normal?
for 218*230image
https://pp.vk.me/c614825/v614825550/1bdc2/-GJFkroeHiQ.jpg

from chainer-fast-neuralstyle.

Related Issues (20)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.