Comments (3)
This is something we're working on. We didn't work on optimizing the memory allocator yet, so large batch sizes or big images that work on CUDA devices may not be supported on DML devices yet, depending on the device. This is also due to the fact that we don't have complete operator coverage yet, so copies between CPU/GPU happen more often than they should, therefore utilizing a lot of memory.
In your case, reducing the batch size when working with bigger images will probably let you run the model without errors.
from tensorflow-directml.
It seems to me that the Conv2D layer has some sort of memory allocation problem, I ran a Dense model with more than 100 million parameters without a problem, but even a 50 thousand params Conv2D model has memory issues, so my conclusion is that maybe some memory leak is happening.
from tensorflow-directml.
We just released tensorflow-directml 1.15.3.dev200911 with many improvements to the memory allocator. You can try it out and tell us how it goes!
Also, since we have now open-sourced our fork, new tensorflow-directml issues should be opened over here.
from tensorflow-directml.
Related Issues (20)
- Not able to use my own callbacks HOT 3
- Tensorflow-directml is not making any difference in processing times in GPU vs CPU HOT 1
- AMD APU support? HOT 2
- Cannot assign a device for operation embedding/embeddings/Initializer/random_uniform/ HOT 5
- directml on custom tensoflow build ? HOT 1
- Windows Camera post process(DMFT) with DirectML(Tensorflow) HOT 10
- It's not working on Intel Graphics 5500 HOT 1
- Use c api to load pb models HOT 1
- how to set the default device using cAPI?
- session run crashed when runing on nvidia gpu HOT 1
- Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. HOT 2
- unbox expects Dml at::Tensor as inputs HOT 1
- Is it not supports the amd gpus without rocm?
- Does directml support multi-GPU training
- TensorFlow-DirectML Does Not Exist HOT 7
- AttributeError: module 'tensorflow' has no attribute 'float32' HOT 1
- CPU instructions notification after installing
- python newer versions support
- tensorflow GPU on WSL2 produces a segmentationfault
- RC astro tool operations not supported in DirectML HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorflow-directml.