Comments (2)
This is not a GRU4Rec related error, but a sign that something in your Theano setup is not correct (it's not even the fault of Theano, but there is some kind of incompatibility between the driver/cuda/cuDNN on your system).
Some ideas on what could have gone wrong:
- Maybe your nVidia driver is too old (as the error message suggests). You should have at least r396.26, but it also works with newer (for example I use r430.14). If your driver is old, try upgrading it. (When upgrading the driver pay attention to the runfile vs. package issue.)
- Maybe the cuDNN you use was built for a different cuda version (there are different cuDNN 7.6 libs built for cuda 9.2, cuda 10.0, cuda 10.2, etc). Double check that your cuda and cuDNN match.
- Maybe when you built libgpuarray/pygpu, you did it in a different environment and it links to a different cuda/cuDNN version than you are using currently.
- It is less likely, but maybe a different cuda version is in your
PATH
or inLD_LIBRARY_PATH
or inCUDA_HOME
(or these environment variables don't point to any cuda version). - You can also try disabling cuDNN usage in Theano by running it with the appropriate flag:
THEANO_FLAGS=dnn.enabled=False python run.py ...
. I don't think that this has a significant impact on the speed of GRU4Rec, but I haven't tried yet. If setting this flag helps, the issue is definitely some kind of incompatibility between your cuDNN and cuda (and/or nVidia driver). - You can also try to delete the theano cache (
theano-cache purge
or manually deleting the contents of~/.theano/
), maybe something is stuck there from a previous version.
If nothing else works, you can set up the whole environment from the ground up. I usually do it this way, because then I know exactly what was installed. It has worked for me 100% of the time. The main steps are:
- Uninstall your current cuda and nVidia driver (if these are installed)
- Install the nVidia driver (>=396.26)
- Install cuda 9.2
- Set the
CUDA_HOME
environment variable to<cuda_install_dir>
(default:/usr/local/cuda-9.2
), and make sure that<cuda_install_dir>/bin
and<cuda_install_dir>/lib64
are added toPATH
andLD_LIBRARY_PATH
respectively. - Download the appropriate cuDNN libs and copy them under
cuda/lib64
- (If you start on a new machine, this is the point where you also install python and the required packages (optionally in pyenv/virtualenv))
- Install pycuda
- Build and install libgpuarray & pygpu
- Build and install Theano
from gru4rec.
Thank you very much.
I will check the problem according to your comments.
from gru4rec.
Related Issues (20)
- hello,i have some questions about the code HOT 2
- About training time HOT 1
- Is it possible to output the embedding of user/session and item vectors? HOT 1
- NOT RNN MODEL HOT 2
- Additional Negative Sampling: Conditional Statement Logic Error HOT 1
- generate_samples function call in gru4rec.py HOT 2
- BPR loss implementation question
- Fit function in gru4rec.py missing data sort HOT 1
- predict_next_batch not considering other products in the same session HOT 2
- (Question) - How to use all items in a session for prediction? HOT 2
- No hidden state reset in get_metrics HOT 4
- Where is the data file ?
- Can you make a brief explaination on how you calculate recall ? HOT 2
- Incremental training (retrain) support removed
- ValueError: Input dimension mis-match. (input[2].shape[0] = 2080, input[3].shape[0] = 32)
- cuda error
- GFF code
- Testing Error:: start = offset_sessions[iters] IndexError: index 2 is out of bounds for axis 0 with size 2
- Evaluating baselines
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gru4rec.