cshjin / microgrid_sim Goto Github PK

First need to get some general ideas of optimization
Probably, stochastic programming is one option.
Also, we can apply most recent methodologies to get out goals.

• Artificial intelligence.
• Data mining.
• Random algorithm.
• Neural network algorithm.
• others.

Oh my god, finally, I realize what I am dealing with.

One of those problems is called unit commitment problem

Get some knowledge about it.

20 years original weather data will have a lot of missing records.

• count each weather type & compare to actual distribution. (# of hours)
• count # of hours do not have MR (<30 mins)
• cautions to datasets size.

Two parts of data need in the model
One is the electricity consumption of a specific location, for example the hourly consumption data of Chicago area.
U.S. Energy Information Administration http://www.eia.gov/

Another data is related to weather, may be both historical and forecast date are need.
National Climate Data Center http://www.ncdc.noaa.gov/

Currently calculation is not accurate at all.

Need IMMEDIATE fix.

It's a bit weird, what I have done is in AMPL-Project. 💢
The repo should be the desired workspace, and also it should be independent enough and well organized.

Anyway, it's half way to be here.

Describe all the possible assumptions we need literally.

RL is directly related to resource management.

Ref:
Reinforcement learning for microgrid energy management

1. what's the scenarios at 2nd stage and 3rd stage
2. what's the probablities of scenarios at 2nd and 3rd stage
3. what's the simulated resources at 2nd and 3rd stage
   1. resources probabilities at 2nd and 3rd stage
   2. resources calculation
4. what's the distribution of scenarios at 2nd and 3rd stage?

Not easy to answer. Review implementation at 2-stage problems.

Properly will use other languages to parse output data.

A typical output of AMPL will be like the following;

t = 1

: _objname   _obj     :=
1   cost     25.09
;

:            _varname          _var _var.lb   _var.ub      _var.rc      :=
1    "amount['BC']"              50     0     Infinity   8.67362e-19
2    "amount['BG']"               0     0     Infinity   0.0102
3    "amount['GB']"               0     0     Infinity   0.004
4    "amount['GC']"             150     0     Infinity   0
5    "amount['RB']"               0     0     Infinity   0.004
6    "amount['RC']"             800     0     Infinity   0
7    "amount['RG']"               0     0     Infinity   0.0102
8    "amount_stage['BC','N']"     0     0     Infinity   0
9    "amount_stage['BC','A']"     0     0     Infinity   0
10   "amount_stage['BC','M']"     0     0     Infinity   0
11   "amount_stage['BG','N']"     0     0     Infinity   0.00714
12   "amount_stage['BG','A']"     0     0     Infinity   0.00204
13   "amount_stage['BG','M']"     0     0     Infinity   0.00102
14   "amount_stage['GB','N']"     0     0     Infinity   0.0028
15   "amount_stage['GB','A']"     0     0     Infinity   0.0008
16   "amount_stage['GB','M']"     0     0     Infinity   0.0004
17   "amount_stage['GC','N']"   200     0     Infinity   0
18   "amount_stage['GC','A']"   600     0     Infinity   0
19   "amount_stage['GC','M']"   800     0     Infinity   0
20   "amount_stage['RB','N']"     0     0     Infinity   0.0028
21   "amount_stage['RB','A']"     0     0     Infinity   0.0008
22   "amount_stage['RB','M']"     0     0     Infinity   0.0004
23   "amount_stage['RC','N']"   800     0     Infinity   0
24   "amount_stage['RC','A']"   400     0     Infinity   0
25   "amount_stage['RC','M']"   200     0     Infinity   0
26   "amount_stage['RG','N']"     0     0     Infinity   0.00714
27   "amount_stage['RG','A']"     0     0     Infinity   0.00204
28   "amount_stage['RG','M']"     0     0     Infinity   0.00102
;

:             _conname           _con.slack _con.dual    :=
1    meetDemand                       0        0.051
2    "meetDemand_stage['N']"          0        0.0357
3    "meetDemand_stage['A']"          0        0.0102
4    "meetDemand_stage['M']"          0        0.0051
5    batteryLimit                     0        0.001
6    "batteryLimit_stage['N']"        0        0.035
7    "batteryLimit_stage['A']"        0        0.01
8    "batteryLimit_stage['M']"        0        0.005
9    resourcesLimit                   0       -0.051
10   "resourcesLimit_stage['N']"      0       -0.0357
11   "resourcesLimit_stage['A']"      0       -0.0102
12   "resourcesLimit_stage['M']"      0       -0.0051
;

1. a proper data storage structure.
2. a gentle data visualization

At lease three type of comparison needed:

• Deterministic model
  • Hour by Hour model solving.
  • Total cost should be the sum of optimal solution
• Two-Stage stochastic programming model
  • 2-Hour by 2-Hour model solving
  • Total cost should be the first-stage cost + the cost of second-stage based on what happened in the following hour.
    
• Repeated Two-stage stochastic programming model
  • repeated 2-Hour by 2-Hour model solving
  • Total cost is the sum of first-stage, since in every second-stage, solve the optimized solution again.
    

Ideally, the third one should be the best. And of course, we need to look a long term result, rather than just think about few hours.

Literally, describe the project much more precisely.
In which perspective? What are assumptions? What objectives? What constrains.

Linux platform is much more efficient than windows?

Why?

Choose a wise simulation tool.
For details:

CPLEX: IBM ILOG CPLEX Optimization Studio
Matlab: Introducing Power Electronics matlab
Python(SimPy): SimPy

1. reduce scenarios to 8
2. kmeans algorithm gets different results from MATLAB and Python.
3. reduce scenarios to 3

Weather data and solar data timestamp match issue.

1. wunderground timestamp are
   1. not recorded every hour
   2. some day record from 1:00AM while others record from 12:00AM
   3. daylight saving shifting issue.
   4. DST (daylight saving time) & UTC (Coordinated Universal Time)
2. NREL data are:
   1. record every hour
   2. no daylight saving shifting
   3. LST (local standard time) ref

Which data to be used in model is quite sensitive.

• clustering into 2, 3, 8, 42
• for each set of scenarios:
  calculate repeating v.s. non-repeating (show actual cumulative cost in addtion to plots)
• look into demand values & parameters:
  !!! CRITICAL !!!
  • why are all 3 values for non-repeating the same?
  • why are all 3 values for repeating the same?
  • why is the repeating worse than non-repeating in all 3 scenarios?
• Implement "long 2nd stage" correctly

Calucation of total cost:
using data from 1991-2010 for calculatng prob. & expectations comparing cost of running the algorithm in 2010.
Also calculate the cost for 2010 if all data was known

Two things need to be improved:

• Error handling
• Multiporcessing

I got a lot of data in plain text, and proper it is not the best way.

Now consider save data into files using like numpy.save functions.

see demo:

1. reorganize workspace
2. import work in AMPL-project
3. improve performance of AMPL script (currently in 2-stages)

Either use shifted multi-stages, or using average multi-stages.
More reference needed.

Function it may apply:
LinearProgramming
Minimize