Comments (4)
Depends on what you're interested in?
The code is really general purpose, there's so much that you can change/play with. Once you internalize it, its cool!
There are parameters which control the
i) The choice of the solver and it's parameters - i.e. stuff which controls the training of the deep neural network
ii) Function approximator - stuff which controls architecture of the network independent of the game/ale - i.e. choice default=cuda_conv, and layers sizes/activation funcs, weight initialization
iii) Function approximator - stuff which controls the game/ale dependent architecture of the CNN which is dependent on the game/ale - i.e. the input/output layers of the CNN, number of actions, the screen input, ...
iva) Preprocessing data - its vital but luckily, Nathan & Alejandro have done all the hard work here
ivb) The memory replay problem - phi length, mini-batches, ...
v) Objective function definition - you might want to redefine the loss function
vi) The RL problem - discount factor, epsilon stuff, ..
vii) Experiment setup parameters - RL-Glue stuff
I think it's best to focus on the things you find most interesting, then adapt/play with the code, and forget the rest?
It helps if you work on one small bit at a time - there's stuff which is generic to deep learning, and stuff which is specific to the deep-Q-RL algorithm, and more stuff which is specific to this implementation, i.e. ALE/RL-Glue.
Have you seen this?
http://cs.stanford.edu/people/karpathy/convnetjs/demo/rldemo.html
If you go through the code of that demo you should get some ideas of how you can understand/play with/try to improve this projects code
There's a forum which discusses these things as well?
https://groups.google.com/forum/#!forum/deep-q-learning
This post by Ilya Sutskever is pretty hot too,
http://yyue.blogspot.ca/2015/01/a-brief-overview-of-deep-learning.html
from deep_q_rl.
Hi @ildoonet . I'm glad you are getting some use out of the package. Here is the logic behind the current organization, such as it is:
cnn_q_learner.py
- This is intended to encapsulate the deep-q-learning algorithm independent of any particular problem domain. Default parameter values here will generally be overriden.rl_glue_ale_agent.py
- This file usescnn_q_learner.py
to create an agent for the ALE domain within the RLGlue framework. If there are agent parameters that I want to be adjustable, I make them command line arguments for this executable, and I try to keep the defaults set to the best known values.
How you manage parameter values, and which parameter values you are interested in, will largely depend on what you are doing with the package. My priority has been on making sure that the parameters I'm most interested in can be adjusted without modifying the source code.
I'm open to (easily-merged) pull requests or suggestions for improvement, but this isn't an issue I have a lot of time to work on. You could also take a look at @alito's fork. He has put some effort into general tidying-up, and he may have a better way of managing default parameter values.
from deep_q_rl.
I'm going to close this, since there are no concrete plans to do anything about it.
from deep_q_rl.
Thanks!
from deep_q_rl.
Related Issues (20)
- What is the memory requirement for this program HOT 3
- Choose action with epsilon set to self.epsilon instead of 1.0 prior to hitting replay size HOT 2
- Memory error, sysmalloc: Assertion failed HOT 3
- ROM file wont load HOT 11
- Reproducibility HOT 4
- Training results don't match Deepmind implementation HOT 7
- "reward_per_epoch" should be "reward_per_episode" in results.csv HOT 1
- Training on nips paper doesn't run
- Gradient zero when diff is clipped HOT 10
- something wrong with referenced Lasagne
- ale-python-interface is not work HOT 1
- cPickle save/load pkl file on Windows
- plot_results.py: IndexError: index 3 is out of bounds for axis 1 with size 2
- ValueError: numpy.dtype has the wrong size, try recompiling
- UnusedInputError
- Issue on Input Shape of network HOT 1
- learning.csv doesn't contain average loss per epoch
- Illegal instruction (core dumped) when loading the ROM
- ImportError: No module named cuda.var in theano version 0.9.0
- Error memory HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deep_q_rl.