Comments (6)
Hi,
To introduce weight normalization (or any other form of normalization such as layer normalization or the usage of SeLu), you would need to modify the neural network (e.g. deer/q_networks/NN_keras.py if you use AC_net_keras). You can then also choose appropriate hyper-parameters that are well suited for the given neural network when you instantiate one of the learning algorithms (or even write your own modifications of the provided learning algorithms).
Hope this helps.
PS: the links you provided do not have the right url.
from deer.
I appreciate your quick response!
Honestly, I was hoping that applying weight normalization to the neural network model provided by NN_keras.py would only require to make changes to AC_net_keras.py in ll. 75-82 similar to the changes made in ll. 63-69 of cifar10_cnn.py in the aforementioned example. As you probably noticed, I'm new to Deep RL. Maybe you can point out the reason(s) for why one would have to change NN_keras.py in order to introduce weight normalization.
from deer.
Right, in the code you provided they have integrated the normalization in the optimizer instead of modifying the neural network definition. So it should be possible to directly take their optimizers and use it in the learning algorithm (replacing the simple SGD+momentum in AC_net_Keras as you pointed out)
from deer.
Thanks for the clarification.
Is normalization and weight normalization in particular something you would like to add to your library at this point? If it is not, you may, of course, feel free to close this issue. Otherwise maybe someone else or I will propose an appropriate implementation and post it here at some point.
Anyway, thanks for your help!
from deer.
We have to be sure that the benefit of adding this outweigh the additional complexity in the code and for the user. If it's only a matter of adding an update rule in AC_net_keras.py in ll. 75-82, then maybe ok. But I guess we would need to add the whole weight normalization code in this library...
from deer.
Yes, I thought about that, as well. Overall, Weight Normalization might not be worth the additional complexity. One would not only have to add the whole code by OpenAI but also find a solution for the data-dependent initialization. The latter would probably make Weight Normalization only applicable to certain environments or even leave the solution of the initialization problem completely to the user.
Therefore, I decided to go with Batch Normalization for now. The implementation for Keras models is very straightforward and training seems to improve. You are welcome to take a look at my implementation of Batch Normalization for NN_keras.py.
PS: I am not sure if feeding the "training=0" argument to the layer is correct but it seems to work.
from deer.
Related Issues (20)
- ReadTheDocs Link Broken HOT 1
- MemoryError on run_PLE.py example HOT 1
- q_networks.AC_net_keras, q_networks.q_network_keras and q_networks.q_network_theano only use 95mb of GPU HOT 1
- Is there any pre-trained model? HOT 1
- TypeError: _buildDQN() takes exactly 2 arguments (1 given) HOT 1
- AC_net_keras qnetwork.getAllParams() HOT 3
- Action limits are getting exceeded HOT 1
- TRPO algorithm HOT 2
- Conv2D channels_last in the Keras HOT 1
- Error for bleeding edge version installation HOT 1
- How to use LSTM? HOT 4
- More information about the LongerExplorationPolicy HOT 3
- DDPG implementation HOT 4
- CRAR continuous action space HOT 2
- Agent not learning for maze environment with CRAR HOT 2
- how can i test my deer model HOT 2
- MG_two_storages HOT 1
- MG example with custom environment HOT 6
- MG two sorages HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deer.