praveen-1 / rl-library Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/rl-library
License: Apache License 2.0
Automatically exported from code.google.com/p/rl-library
License: Apache License 2.0
On useful visualization component would be to show the value of all the
next actions.... this would allow the user to visualize things more easily.
However, perhaps this should be an agent simulator....
It could be a generic component, like the value function visualizor. I
guess you could literally have people turn on and off different
visualizations (in the viz program), like turning on or off a monitor of
various conditions. Basically, you could make it like a "build you own"
visualizer. Hmm.
Original issue reported on code.google.com by [email protected]
on 26 Jul 2007 at 1:56
We should work a little harder on designing EnvironmentShell.EnvLoadingHelper
I Want 2 of them. One that lists and loads environments from the local
file system. The other will go onto the web. This works and there is an
example of it in there after an if statement that is false.
Original issue reported on code.google.com by [email protected]
on 26 Jul 2007 at 7:11
Need to talk to Matt Taylor and get Soccer integrated.
Original issue reported on code.google.com by [email protected]
on 9 Oct 2007 at 10:29
We should create a Python Trainer that does exactly what the Java and CPP
trainers do.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 11:23
We should add value function components and turn the value function slider
on and off depending on if the agent supports it.
Original issue reported on code.google.com by [email protected]
on 9 Oct 2007 at 10:16
When I run MountainCarAgent against consoleTrainerJava, sometimes I get
double free errors. I think it might be that cleanup() is or isn't being
called.
Should look into that, cause it looks bad.
RL_agent(7914) malloc: *** Deallocation of a pointer not malloced:
0x3003e0; This could be a double free(), or free() called with the middle
of an allocated block; Try setting environment variable MallocHelp to see
tools to help debug
RL_agent(7914) malloc: *** Deallocation of a pointer not malloced:
0x300400; This could be a double free(), or free() called with the middle
of an allocated block; Try setting environment variable MallocHelp to see
tools to help debug
RL_agent(7914) malloc: *** Deallocation of a pointer not malloced:
0x3003e0; This could be a double free(), or free() called with the middle
of an allocated block; Try setting environment variable MallocHelp to see
tools to help debug
RL_agent(7914) malloc: *** Deallocation of a pointer not malloced:
0x300400; This could be a double free(), or free() called with the middle
of an allocated block; Try setting environment variable MallocHelp to see
tools to help debug
Original issue reported on code.google.com by [email protected]
on 12 Oct 2007 at 10:42
I thought I made this issue but I Can't find it.
I'd like the RL-Glue directory in the root of the training pack to actually
be an svn:externals to the RL-Glue repository. I don't like the idea of
maintaining 2 copies of that code.
Adam has found some debug spew we need to clean up, and I think I have too
-- the JAVA rl glue seems to be printing task specs out.
Assigned to Mark Lee b/c he's the RL-Glue and SVN expert.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 9:50
We need a sample agent for the Polyathalon
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 11:21
The value function resolution slider should only be there if you're using
it. Not sure if that functionality should be moved into the visualization
somehow (maybe?) or what, but we need to fix it for the competition.
Original issue reported on code.google.com by [email protected]
on 13 Sep 2007 at 12:44
In RLVizLib, there is an interface that is called QuerableAgent, it should
be Queryable agent. We should fix this and propagate the changes to all of
our docs and sample agents, and the viz components and messages that use this.
Original issue reported on code.google.com by [email protected]
on 5 Oct 2007 at 8:57
Probably every single file we send out should have ownership and copyright
messages, as well as a license.
We should get on settling what this will be.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 7:31
We should find a way that we can take a C or C++ env/agent and build it
into a dylib without having to edit it and extern "C" anything.
Original issue reported on code.google.com by [email protected]
on 14 Sep 2007 at 5:32
Should be a simple way to decide if agents/envs or both will be loaded
through the visualizer.
Original issue reported on code.google.com by [email protected]
on 13 Sep 2007 at 12:45
We need the Helicopter Sample agent working
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 11:22
Need fault tolerance in the protocol. Need a query system to see if the
agent or environment supports it, and everthing needs to work well if they
don't.
Original issue reported on code.google.com by [email protected]
on 25 Jul 2007 at 11:50
This is a big one.
Original issue reported on code.google.com by [email protected]
on 9 Oct 2007 at 10:28
This is a big issue and it needs its own wiki page and people working on
it, but we need to write the phone home server or scripts.
Original issue reported on code.google.com by [email protected]
on 9 Oct 2007 at 10:21
We need a simple idea of what the Polyathalon domain might look like from
the user perspective.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 11:22
We need a sample agent for mountain car, I think the plan was to make it a
Java Sarsa(lambda) tile coding agent.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 11:21
In Tetris, its important to tell the agent the number of Rows and Columns.
We've decided it makes sense to use the messaging system for this. So:
1) The benchmark needs a way to ask the environment for the rows and cols
2) The benchmark needs to send these to the agent
3) The agent needs to parse these
We should be able to do this on the environment side using RLViz messages,
and on the agent side using RLViz messages on the Java side and some
simple C/Python code otherwise.
Original issue reported on code.google.com by [email protected]
on 10 Oct 2007 at 7:41
Make a way for the Java environments for the competition to take a
parameter configuration number instead of actual parameters.
Original issue reported on code.google.com by [email protected]
on 9 Oct 2007 at 10:21
While neither Querable or Queryable are real words, the latter is in more
common use by programmers.
The place I found it was
rlVizLib.messaging.interfaces.getEnvMaxMinsInterface.getMaxValueForQuerableVaria
ble()
Original issue reported on code.google.com by [email protected]
on 5 Oct 2007 at 8:59
The local glue code is cobbled together and needs to be written right and
tested. IT doesn't implement everything, and I'm not sure its doing the
right thing in terms of time step counts and such. Should look at the core
glue code and compare.
Original issue reported on code.google.com by [email protected]
on 13 Sep 2007 at 1:11
Title says it all
Original issue reported on code.google.com by [email protected]
on 13 Sep 2007 at 5:52
There should be a folder of jars and .dylibs (maybe just jars) for
visualizing and all of the visualizer specific code should be OUT of the
source for bt-Viz and the VizLib. They should be independent, either
inside the environment project or someplace else.
Original issue reported on code.google.com by [email protected]
on 18 Sep 2007 at 4:58
This is a big issue and will take lots of work, but we need the Java tester
program.
Original issue reported on code.google.com by [email protected]
on 9 Oct 2007 at 10:23
Should investigate on how to make all of the connectivity stuff faster.
Not sure if that should be via compressions, or being judicious, but I'd
like the visualization to be even faster and able to handle higher
resolution of value functions.
Original issue reported on code.google.com by [email protected]
on 25 Jul 2007 at 11:48
Right now the consoleTrainers don't really unload, they just load and load
and load again. Is this bad?
Look for memory leaks or other bad possibilities.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 11:11
We had the viz frame working in the tech dist, we should be able to do it
for the competition package... but its not the highest of our priorities.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 7:58
We need to make sure helicopter is working right and we need to contact
Pieter Abeel and make sure we have a good altered problem plan.
Original issue reported on code.google.com by [email protected]
on 12 Oct 2007 at 5:05
We need a functioning trainer that handles agents and can be used to train
agents on the training version of the competition environments.
Original issue reported on code.google.com by [email protected]
on 26 Jul 2007 at 1:50
Shivaram is working on this
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 11:21
The C/C++ files we need (ParameterHolder et. al?) probably shouldn't exist
both in the rl-competition repository and in the library. We should decide
which is the "right" copy (probably the library) and then only point to it.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 10:27
For now, we are going to crash with a stack trace if invalid actions are
selected. In future (competition) we should keep running more robustly.
Original issue reported on code.google.com by [email protected]
on 3 Aug 2007 at 6:30
Need to get the RTS Java visualizer integrated
Original issue reported on code.google.com by [email protected]
on 9 Oct 2007 at 10:29
This tester is also an experiment program. It will be Java based and will
phone home first and load the experiment Jar over the Internet.
Maybe we can do some online scoreboarding as this thing runs and show other
people's results as it goes, with bar graphs or something. What I mean is
that as its benchmarking you could literally have it now how well its doing
which would make it fun to watch.
The phoning home will need to be network code written independent of the glue.
Original issue reported on code.google.com by [email protected]
on 26 Jul 2007 at 1:54
I created a mechanism for asking the environment for his RLVizVersion, and
I stored it in RLGlueLogic.
We need a good way to query this before starting down paths that will make
us send queries that won't be answered.
Original issue reported on code.google.com by [email protected]
on 26 Jul 2007 at 7:41
Java strings are immutable, and if you are building big ones they are horribly
slow. So, to serialize
1600 observations that have 2 doubles each, building immutable strings took
2182 ms. This is
compared to 240 ms to actually send the message over the network and receive a
response.
After changing from immutable strings to using a stringbuffer and calling
append, the time
dropped from 2182 to 7 ms.
That's crazy.
Original issue reported on code.google.com by [email protected]
on 23 Jul 2007 at 3:34
We should create a super simple experiment program in C that could be used
as the basis for making a trainer. This will show people that it really is
cross platform and doesn't require java.
Original issue reported on code.google.com by [email protected]
on 26 Jul 2007 at 1:52
1) We should figure out exactly what characters that break the parameter
Holder (like currently, having :," ", or "_" in either the names of the
parameters or their values (if using strings) might break things.
So, I propose a few changes
a) When parsing messages in RL-Viz, we shouldn't split on spaces all the
time, we should use spaces to split, TO, FROM, CMD, etc... but when we get
to VALS (payload) we should just take the whole rest of the string. This
means that spaces in the payload won't break the payload parse. This
should be one change in GenericMessage.
b) In ParameterHolder, when someone sets a variable name or a Strign
parameter, we should EXPLICITLY check right then for "_" and ":" and throw
an exception or die if we can't use it. Or else we can replace it with
another character like ; and print a warning. If we do this on both sides,
then invalidly specified parameters will work without crashing
c) Test all of this and make sure we can't break it.
Original issue reported on code.google.com by [email protected]
on 30 Jul 2007 at 6:25
There are various updates that need to be made to the competition website,
not sure what they all are, but that's important.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 10:16
Find out about signing jars and obfuscating source or class files.
See what we can do about building an appropriate manifest too, with
copyright and credits.
Original issue reported on code.google.com by [email protected]
on 12 Oct 2007 at 4:40
Right now someone using a value function might have to write like 10 lines
of code to get the slider up and running. We should look at folding all of
that code into something inside the value function code, which lives in the
VizLib.
Original issue reported on code.google.com by [email protected]
on 12 Oct 2007 at 12:53
Each of the projects now can be built with Ant scripts, we should write
some more to do the whole things at once... the equivalent of clean-all
make-all
Original issue reported on code.google.com by [email protected]
on 26 Sep 2007 at 4:14
Make sure everything works in Linux
Original issue reported on code.google.com by [email protected]
on 10 Oct 2007 at 6:26
Here's what would be cool.
A bash script, which when run, goes through a bunch of scenarios, like
running a bunch of combinations of agents and trainers, such that in 60
seconds, one of us can verify that the combinations of things all work.
It's not perfect, but it would help.
Original issue reported on code.google.com by [email protected]
on 12 Oct 2007 at 9:26
the Java Task Spec Parser is Broken. Currently we are hard coding values
for each environment in. BOOOO
Original issue reported on code.google.com by [email protected]
on 27 Jul 2007 at 5:14
I've got an environment where actions are [-5,5]. The Random Agent only
chooses actions from [0,5].
Original issue reported on code.google.com by [email protected]
on 5 Oct 2007 at 9:17
We're pretty desperate for documentation.
Someone needs to put together a getting started guide and get it online.
This has to wait a few days, until the trainers and how to run them settles
a bit.
Original issue reported on code.google.com by [email protected]
on 11 Oct 2007 at 7:32
Fix this.
Original issue reported on code.google.com by [email protected]
on 2 Aug 2007 at 5:12
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.