Default parameters values are defined in multiple source files about deep_q_rl HOT 4 CLOSED

Depends on what you're interested in?

The code is really general purpose, there's so much that you can change/play with. Once you internalize it, its cool!

There are parameters which control the

i) The choice of the solver and it's parameters - i.e. stuff which controls the training of the deep neural network
ii) Function approximator - stuff which controls architecture of the network independent of the game/ale - i.e. choice default=cuda_conv, and layers sizes/activation funcs, weight initialization
iii) Function approximator - stuff which controls the game/ale dependent architecture of the CNN which is dependent on the game/ale - i.e. the input/output layers of the CNN, number of actions, the screen input, ...
iva) Preprocessing data - its vital but luckily, Nathan & Alejandro have done all the hard work here
ivb) The memory replay problem - phi length, mini-batches, ...
v) Objective function definition - you might want to redefine the loss function
vi) The RL problem - discount factor, epsilon stuff, ..
vii) Experiment setup parameters - RL-Glue stuff

I think it's best to focus on the things you find most interesting, then adapt/play with the code, and forget the rest?

It helps if you work on one small bit at a time - there's stuff which is generic to deep learning, and stuff which is specific to the deep-Q-RL algorithm, and more stuff which is specific to this implementation, i.e. ALE/RL-Glue.

Have you seen this?

http://cs.stanford.edu/people/karpathy/convnetjs/demo/rldemo.html

If you go through the code of that demo you should get some ideas of how you can understand/play with/try to improve this projects code

There's a forum which discusses these things as well?

https://groups.google.com/forum/#!forum/deep-q-learning

This post by Ilya Sutskever is pretty hot too,

http://yyue.blogspot.ca/2015/01/a-brief-overview-of-deep-learning.html

from deep_q_rl.

Hi @ildoonet . I'm glad you are getting some use out of the package. Here is the logic behind the current organization, such as it is:

• cnn_q_learner.py - This is intended to encapsulate the deep-q-learning algorithm independent of any particular problem domain. Default parameter values here will generally be overriden.
• rl_glue_ale_agent.py - This file uses cnn_q_learner.py to create an agent for the ALE domain within the RLGlue framework. If there are agent parameters that I want to be adjustable, I make them command line arguments for this executable, and I try to keep the defaults set to the best known values.

How you manage parameter values, and which parameter values you are interested in, will largely depend on what you are doing with the package. My priority has been on making sure that the parameters I'm most interested in can be adjusted without modifying the source code.

I'm open to (easily-merged) pull requests or suggestions for improvement, but this isn't an issue I have a lot of time to work on. You could also take a look at @alito's fork. He has put some effort into general tidying-up, and he may have a better way of managing default parameter values.

from deep_q_rl.

I'm going to close this, since there are no concrete plans to do anything about it.

from deep_q_rl.

Thanks!

from deep_q_rl.