Comments (9)
Hi
You can change the display_size in the config file to 4 or 2, and try again.
from munit.
@visonpon I couldn't fix this. Could you please provide details? I have the same problem and I'm new to torch.
UPDATE: Ok I got it, just read the cmd error output of pytorch, it tells what to do.
Thx.
from munit.
@WuChanghao233 Still... i change display_size in the config file (changed to 2)
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1524580938250/work/aten/src/THC/THCTensorRandom.cu line=25 error=2 : out of memory
Traceback (most recent call last):
File "train.py", line 5, in
from utils import get_all_data_loaders, prepare_sub_folder,
File "/root/model/utils.py", line 5, in
from torch.utils.serialization import load_lua
File "/root/anaconda2/lib/python2.7/site-packages/torch/utils/serialization/init.py", line 2, in
from .read_lua_file import load_lua, T7Reader
File "/root/anaconda2/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 184, in
register_torch_class('Storage', make_storage_reader)
File "/root/anaconda2/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 181, in register_torch_class
reader_registry[cls_name] = reader_factory(cls_name)
File "/root/anaconda2/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 160, in make_storage_reader
element_size = python_class().element_size()
File "/root/anaconda2/lib/python2.7/site-packages/torch/cuda/init.py", line 492, in _lazy_new
_lazy_init()
File "/root/anaconda2/lib/python2.7/site-packages/torch/cuda/init.py", line 161, in _lazy_init
torch._C._cuda_init()
RuntimeError: cuda runtime error (2) : out of memory at /opt/conda/conda-bld/pytorch_1524580938250/work/aten/src/THC/THCTensorRandom.cu:25
from munit.
Same problem occurs to me, my GPU has 11GB memory, but it can't train nether either.
from munit.
Same problem occurs to me, my GPU is GTX 1050Ti (4GB memory).
I try display_size in the config file to 4 or 2, not solve it.
from munit.
i have also encountered this problem , it seems like the part of write_image script lead to this bug since when i commented it , everything is ok. but i don't know how to fix it when i want to see the result during training @mingyuliutw
from munit.
solved it by adding with torch_no_grad
:
from munit.
solved it by adding
with torch_no_grad
:
Can you tell me where to add it?
from munit.
Adjusting the .yaml to shrink/simplify the network works. I imagine the only other options are to optimize their code for memory usage or get a better GPU.
from munit.
Related Issues (20)
- How to implement MUNIT with K Fold Cross Validation?
- Question about Multi-GPU training on single machine HOT 4
- Can you use non-standardised dataset for training?
- Checkpoint images examples?
- Can i do train grayscale-imageset?
- Pytorch version >=1.0 How to load VGG16 pretrain in VGG16.T7? HOT 1
- Experiment of using Instance Normalization vs Layer Normalization on Decoder HOT 8
- AdaptiveInstanceNorm2d and the LayerNorm misunderstand
- May I ask if everyone will prompt "Warning: NaN or Inf found in input tensor." during runtime? HOT 1
- Questions about batch_size and GPU memory usage
- loss Nan
- When are you planning to make it public?
- After F(x)+x, the ResBlock seems not follow by a non-linearity activation HOT 1
- Ideas to speed up training phase
- Missing max pooling layer in the Vgg16 network structure
- outputs = (outputs + 1) / 2. in test code HOT 1
- inception checkpoint? HOT 3
- Ŕ
- summer2winter_yosemite checkpoint?
- Pretrained Inception Network HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from munit.