Comments (5)
Hi Felix,
I should have added the hyperparameters to the readme file (which I will very soon).
CMU:
THEANO_FLAGS='device=cuda0,floatX=float32' nice -n 9 python -u gcnmain.py -hid 300 300 300 -bucket 50 -batch 500 -d ./data/cmu/ -enc latin1 -mindf 10 -reg 0.0 -dropout 0.5 -cel 5 -highway
NA:
THEANO_FLAGS='device=cpu,floatX=float32' python -u gcnmain.py -hid 600 600 600 -bucket 2400 -batch 500 -d ~/data/na/ -mindf 10 -reg 0.0 -dropout 0.5 -cel 15 -highway
WORLD:
THEANO_FLAGS='device=cpu,floatX=float32' python -u gcnmain.py -hid 900 900 900 -bucket 2400 -batch 500 -d ~/data/world/ -mindf 10 -reg 0.0 -dropout 0.5 -cel 5 -highway
Why Twitter-WORLD is smaller in size than Twitter-US?
world has higher number of users, but lower number of tweets per user, so NA is actually bigger in dataset size.
Also note that the random seeds are changed so you might not get the exact results (unfortunately) but in several runs it might be a little better or worse, but in general comparable.
Don't hesitate to contact me if there were more issues.
Afshin
from geographconv.
Hi Afshin,
Thank you for your instant response.
I ran the first command:
THEANO_FLAGS='device=cuda0,floatX=float32' nice -n 9 python -u gcnmain.py -hid 300 300 300 -bucket 50 -batch 500 -d ./data/cmu/ -enc latin1 -mindf 10 -reg 0.0 -dropout 0.5 -cel 5 -silent -highway
However, I only got
dev: Mean: 536 Median: 100 Acc@161: 55
test: Mean: 561 Median: 96 Acc@161: 54
I ran 10 other runs with different seeds (from 1 to 10), but I still can't get an acc@161 higher than 55.
Here is the log I got from these 10 runs:
https://gist.github.com/felixgwu/8ae4c6e7a887092ae30c82fea6d6db40
I wonder if I made some mistakes.
Here is how I create the environment.
I first create a new environment using the requirements.txt file
conda create --name geo --file requirements.txt
However, I got an error when it tried to import lasagne (version 0.1).
The error occurred at:
from theano.tensor.signal import downsample
It seems that theano doesn't have downsample in theano.tensor.signal, so I upgrade both theano and lasagne to the newest version with this command:
pip install --upgrade https://github.com/Theano/Theano/archive/master.zip
pip install --upgrade https://github.com/Lasagne/Lasagne/archive/master.zip
After that I can run the gcnmain.py script without errors.
BTW, I use CUDA 8.0.
I only have limited experience with Theano. Maybe there is something wrong here.
-Felix
from geographconv.
Hi Felix,
Regarding the Lasagne and Theano update, you're right, they should be upgraded.
Regarding running with default hyperparameters, we shouldn't do that because the default hyperparameters are not suitable for all the three datasets, they're just there so that the code runs (e.g. the default hidden layer is only 100 and the bucket is 300 which should be 300 300 300 and 50, respectively for cmu).
Regarding when you run with the command
THEANO_FLAGS='device=cuda0,floatX=float32' nice -n 9 python -u gcnmain.py -hid 300 300 300 -bucket 50 -batch 500 -d ./data/cmu/ -enc latin1 -mindf 10 -reg 0.0 -dropout 0.5 -cel 5 -silent -highway
I had this mistake of putting -silent in the previous command that I put here, If you remove it you'll see the correct results like this: https://gist.github.com/felixgwu/8ae4c6e7a887092ae30c82fea6d6db40
That's why you only got dev: test:
Don't hesitate to send me feedback if something was still wrong, I'd love to fix errors, help.
Thanks Felix.
Afshin
from geographconv.
Hi Afshin,
Thank you so much!
I can finally reproduce your results in the paper on GEOTEXT.
At first I couldn't reproduce it even with the correct command, but after I removed the data/cmu/dump.pkl
and data/cmu/vocab.pkl
and ran the script again, I got the correct results.
I wonder if there is something cached here so that I have to delete them every time whenever I use a different set of hyperparameters.
Here is the log in case someone else also wants to reproduce it.
https://gist.github.com/felixgwu/8c74a28b040b95635fcf28c5c1e3e078
I'll try the other two larger datasets and hopefully, I can reproduce them.
BTW, there are two typos in the command for NA and World datasets in the README file.
~/data/na/
should be ./data/na/
and ~/data/world/
should be ./data/world/
-Felix
from geographconv.
Hi Felix,
Great News.
After running for the first time, the code saves the preprocessed dataset in dump.pkl in the dataset dir. Next time it loads that by default. If that file is made with incorrect hyperparameters, it'll still load it even if the new hyperparameters (e.g. bucket size) is correct. To stop it from doing that we can use -builddata option to force it to reproduce dump.pkl.
Thanks a lot Felix for the typo fixes and all the other help (I'll add them to the repo asap). It made the code easier for everyone else to reproduce.
Afshin
from geographconv.
Related Issues (6)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from geographconv.