gsavarela / ilurl Goto Github PK

Methodology:

For 5 iterations do:
1. Stop learning.
2. For 5x Evaluate 9000 secs. or 100 cycles on static datasets. (RUN on the background)
3. Train for 9000 secs. or 100 cycles.

• DPQ: verify that on the base case (Q(s, a) = 0 for all s and a) the actions are random.
• Make frozen data sets
• Update models/train.py
• Save file on a special folder.
• Results from evaluations must be saved on Root

As of now we only look at half a length of incoming edges. We should be able to look over both incoming and outgoing (without overlap).

The rewards should be computed using speed and count averaged out.

A bug was causing the vehicles to vanish from simulations involving the TrafficLightGridEnv class. The problem was a faulty re-routing strategy on additonal_command() method call. The solution found was to use "copy and paste" inheritance copying the method's implementation provided by AccelEnv class.

A better solution would be to use decorator polymorphism in order to enforce the decorated methods to implement sometimes behaviour from AccelEnv (additional_command())and other times TrafficLightGridEnv (_apply_rl_actions() ) and even both ( init())

SEE:
https://www.codementor.io/sheena/advanced-use-python-decorators-class-function-du107nxsv

for suggestions

Currently sumo supports both inflows which are actually generators of trips and direct uploads of routes files. This issue regards generating a one by one vehicle trip from the inflows files and loading to experiment.

Perform a Komogorov Smirnov test and show that they indeed come from the same distribution.

Use real world intersection (kind of "Scenario") from city of Lisbon -- structural data.

Use "espira" data (vehicle count) to infer the arrival times of vehicles in the intersection. -- conjectural data.

gsavarela@hype13 ~/W/p/i/ilurl> python examples/smart_network.py
/Users/gsavarela/.pyenv/bin/versions/ilurl/lib/python3.6/site-packages/pandas/core/indexing.py:376: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
self.obj[key] = _infer_fill_value(value)
/Users/gsavarela/.pyenv/bin/versions/ilurl/lib/python3.6/site-packages/pandas/core/indexing.py:494: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
self.obj[item] = s

Remove experiments folder in favor of examples folder (as of flow).

Update directories to follow flow standards. e.g
experiments -> examples
envs/agents -> envs/green_wave_env
ql/ -> core/ql/

Recursively rename directories from ilu to ilurl (then we can integrate with other ilu participants)

Agent has been having trouble learning -- the data from incoming & outgoing edges are way too aggregated for the agent to properly adapt.

According to El-Tantawy, 2014 lanes must be separated by components such as the ones controlling north-south and east-west

Create a fully customizable run script in which we may define:

1. Params: (INPUTS)

• EnvParams
• SumoParams
• QlParams
• NetParams
• ExpParams
• InFlow

2. Metadata:
   Generate a description for:

• user
• email
• commit id
• date and time
• scenario generated
• elapsed time
• copy of the run script

3. Generate aggregate performance data: (OUTPUTS)

• emission.csv
• info.json
• Average travel time
• Wait time
• Flow per intersection
• Average outflow
• Queue size

4. Publish overview (INTER EXPERIMENT)

Define a table (spreadsheet) holding the experiments, the results over the aggregate performance statistics.

gsavarela@hype13 ~/W/p/i/ilurl> python examples/smart_network.py
/Users/gsavarela/.pyenv/bin/versions/ilurl/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2957: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/Users/gsavarela/.pyenv/bin/versions/ilurl/lib/python3.6/site-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)

Flow's parameters are mixed on models/train, models/validation, scenarios, and enviroments. Push every class to the parameters module where just QLParams exist.

Add base scenario and networks are intertwined add loaders/networks and place low-level getters e.g get_nodes, get_routes and get_edges to the loaders module alongside with other data stuff.

An installation script is necessary for code-deployment on cluster

Facilitate distribution

Evaluating multiple settings ("examples") are now very cumbersome -- as it all happens by updating the experiments/smart_grid.py file.

Furthermore results are not being properly stored for further comparison.

Segregate scripts into training and evaluation

Train: Set up an example (Scenario+Environment+Sumo+Learning)
Run an experiment (+env)

Eval: Load an environment
Run an experiment (+env)

Create a file with settings ( preventing from having to store a script) of the parameter used withing the evaluation and training?

Real world networks have been added: intersection and grid

Use flow to add:
-Flows
-Routes
-Vehicles

States breakdown helper

Statistical significance of the results

• Veritify that duration and sensor memory is cleared when TrafficLightQLEnv resets

• Using uniform per lane distribution 0.25 * num_lanes train for 10 iterations

• Perform histogram of returns and split distribution