General workflow for simulations:

• Activate virtual environment ($ source env/bin/activate) to make sure python modules are loaded
• Set environment variables for the input data file which we're using, and whether we're reading off a subset of nodes
  • export INPUTFILE=inputData/pricedata_LMP.csv (if using entire raw dataset)
  • export STOPNODE=0 (if using all nodes, or the index of the desired stop node)
  • export NODELIST=TRUE (if using a subset of nodes, generated by AggregatingData.ipynb)
  • export ENDDATE="12/31/16 23:00" (or set the desired end date. Default is "1/31/12 23:00"
• Run efficiency sweep with $ nohup python efficiencySweep.py
• monitor progress with $ top -u emunsing

Outputs:

• Data/efficiencyResults_pidXXXXXtemp.csv : Interim results for thread XXXXX containing cycle count and storage profit/kWh for each node in the batch, a$
• Data/efficiencyPower_pidXXXXXtemp.csv : Interim results for thread XXXXX containing purchases (positive) and sales (negative) for each node (rows) at $
• Data/kwhValue_step_02.csv: Full results for storageProfit for each node (rows) at a range of efficiencies (columns).
• Data/powerOutput_90pct.csv : Full results for purchases/sales of all nodes (rows) at 90% efficiency for all hours in the study period (columns).
• Data/cycleCount_step_02.csv: Full results for cycleCount for each node (rows) at a range of efficiencies (columns).

The system will likely hang for some cores at some point in the process. These jobs will not complete, but instead will choke the thread (sorry). Eventually, will need to kill with $ kill -9 ##### where ### is the process id (from $ top -u emunsing)

After killing frozen processes, can get a list of the remaining nodes to process by running AggregatingData.ipynb to create nodeList.pkl file. This will become the preferred source of data if the environment variable NODELIST is set to TRUE.

Files:

• simulationFunctions.py : optimization heavy lifting and helper functions
• efficiencySweep.py : manage the dispatch of data into parallel processes
• AggregatingData.ipynb : collect data from different parallel processes into a cohesive whole
• Implementing CyLP_APEN.ipynb - outdated, early attempt