Issue

Before the serialization feature was added, the documentation specification was shortened by deleting imported modules at the package level. For example, in events/__init__.py, the lines

del event
del event_queue

were included in the file so that, for the PluginEvent documentation, the actual specification only appeared for acnsim.PluginEvent, not acnsim.event.PluginEvent. The serialization feature required complete specification of objects' paths, so deleting intermediate modules wasn't allowed. There is probably a way to specify (in the documentation files themselves) which parts of the package tree to include in the documentation; we should make the necessary changes to prevent documentation redundancies.

Proposal

ACN-Sim should include functions similar to caltech_acn() (see #14) for JPL and office001.

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of acnportal.

Describe the bug
scikit-learn is supposed to be an optional dependency for acnportal, however it raises an error if you try to import acnsim without having it installed.

To Reproduce
from acnportal import acnsim

Expected behavior
Does not throw error.

Screenshots

ImportError: scikit-learn is required to use the stochastic_events module. Please install scikit-learn using
	pip install scikit-learn
or install acnportal with extra options
	pip install acnportal[scikit-learn]

Is there any minimum Python version required for ACN portal? Clearly, Python 3.6 is necessary for using f-strings. But ideally, if you target Python 3.7+, then you can simplify a lot of the typing annotations. For example, you can do things like the following in Python 3.7+:

from __future__ import annotations

def f(x: dict[str, int] | None = None) -> dict | None:
    return x

I didn't see any minimum Python version specified in the docs, so if that can be added (and ideally Python 3.7+), then I could help add appropriate typing annotations.

Issue

It can be helpful to see the extent to which constraints were violated when a constraint-violating schedule is detected by the Simulator.

Solution

The warning message displayed when a constraint violation occurs should include the maximum constraint violation over all constraints and all times so users can get an estimate for how severely constraints were violated. Also, the tolerance in is_feasible should be settable by the user. For example, a user may tolerate a 1e-3 violation in certain algorithms, notice that the warnings thrown by the Simulator show max violations below this threshold, and then decide to increase the constraint violation threshold to this value in subsequent simulations.

Issue

Initially I implemented pilot_signals and charging_rates as dictionaries of lists. However, this is highly inefficient as most operations can be vectorized for major speedups. In fact we already convert these lists into matrices in multiple functions such as constraint_current() within ConstraintSet.

Decision

We will need to decide if whether to use np arrays or pd DataFrames for this task. Arrays will likely be a bit faster and use less memory, but DataFrames have the benefit of naming rows directly (helpful for keeping track of which EVSE each row corresponds to and the ability to use time as the index, which may be helpful.

Also, for fixed duration experiments we can allocate the array size at the beginning of the simulation, however if we want to support interactive simulations, we need to support dynamic resizing of these arrays.

To-Do First

Before undertaking this issue, tests should be written for all methods which interact with pilot_signals and charging_rates.

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of acnportal.

Describe the bug
[A clear and concise description of what the bug is.
Check out this blog post for some suggestions on writing bug reports.]

To Reproduce
[Steps to reproduce this bug.]

Expected behavior
[A clear and concise description of what you expected to happen.]

Screenshots
[If applicable, add screenshots to help explain your problem.]

System Info

• OS: [e.g. macOS 10.15.3]
• acnportal version [e.g. 1.0.3]
• dependency versions

Additional context
[Add any other context about the problem here.]

Issue

Many use cases involve running multiple simulations with different timeframes, algorithms, or parameters. These simulations can be run in parallel to save researchers lots of time. We should build tools to make running parallel simulations easy.

Issue

The SYM COMP current unbalance in analysis gives NaNs for certain inputs. We should figure out how to account for these.

Issue

Currently CaltechACN is a subclass of ChargingNetwork. However, CaltechACN only configures the network in the init() function. Instead, CaltechACN should be a function which returns a properly configured ChargingNetwork.

This is especially helpful since now CaltechACN and other site configurations can be called on subclasses of ChargingNetwork.

Issue

Many use cases for ACN-Sim involve running hundreds to thousands of simulations, i.e. comparing algorithms, tariff structures, months, sites, or demand distributions. Since these simulations are independent, they can be run in parallel. I have been using multiprocessing to run 4 processes in parallel on a server, but with AWS Lambda functions we could run thousands of simulations in parallel! This means experiments that used to take days could be done in less than 15 minutes.

References

http://pywren.io

Issue

We might want to test feasibility of multiple schedules using one function call (which remains fast if we use matrix multiplication). We should add this functionality under a new function name.

The reset function in the EV class attempts to call the battery's reset method without providing an initial charge to reset to, which is a required argument.

Proposed solutions

We could either:

• Instead of calling the battery's reset function, we could store a copy of the initial state of the battery on initialization of the EV and use this to reset the battery.
• specify that the battery should be reset to some specified initial state of charge.

I tend to prefer the former. We'll also need to update the tests for EV.

In order to provide more information about the battery fitting function, batt_cap_fn, in battery.py, we should provide a writeup of the method and add a link in the description.

Issue

Storing schedule_history in memory could take a lot of RAM.

Idea

Move schedule_history to file periodically. Perhaps use asyncio.

The signals.tarriffs submodule includes several tariffs as JSON files. However, when installing with pip, these files are included, so TOUTariffs throws an error. The current method of adding the files to package_data in the setup.py file does not work.

Current workaround is to clone the repo then install with
pip install -e .

Is your feature request related to a problem? Please describe.
As a acnportal user, I would like to be able to directly install acnportal via pip using PyPI, so that I don't need to clone the repository.

Describe the solution you'd like
On each release, we should use GitHub actions to push a new version to PyPI.

Issue

With a matrix implementation of network constraints, ConstraintSet and Constraint classes become unnecessary.

Proposed solution

We will replace ConstraintSet and Constraint classes with instance variables in the Network class: one is a numpy matrix of constraints, another is a numpy vector of limits on the constraint currents.

Additional considerations

If pandas can be made to add Series such that if a column exists in both series, the associated values are summed, whereas columns in only one of the two series remain the same, the Current class becomes unnecessary, so the constraint_set.py file can be removed. We will need to explore a way of executing this definition of sum natively in pandas.

Implementing constraint_current will either require boolean indexing of the constraint matrix or padding of the input load currents so multiplication with the unaltered constraint matrix is possible. Both of these methods should be performance tested.

As numpy arrays don't have information on row indices (unlike pandas) we should store a dictionary mapping indices to evse ids as part of the network.

We will consider adding functionality to convert the constraint matrix into a pd DataFrame and return it for readability.

EV.reset() calls Battery.reset() which takes an initial charge argument. Currently, EV.reset() doesn't provide this argument, causing a bug. We should decide if reset should instead reset battery to an initial SOC rather than an input initial charge.

Issue:
Currently, allowable_pilot_signals only supports EVSE and FiniteRatesEVSE. We should modify the output of allowable_pilot_signals to include the deadband constraint and test that all algorithms work with DeadbandEVSEs.

Feature

In some experiments it is helpful to not assign users to spaces deterministically, but rather according to some probability distribution over the open spaces in the network.

In the case that no spaces are available, the EV should be placed into a queue and wait for the next available space (possibly with a delay to model switching time).

Is your feature request related to a problem? Please describe.
Currently the plugin() method for ChargingNetwork takes in an EV and a station_id. This could be confusing if station_id does not match ev.station_id.

Describe the solution you'd like
To fix this, I propose that we remove station_id as an argument for plugin(), instead we will automatically plug in the vehicle to the EVSE in ev.station_id. If that EVSE is taken, we will raise an error.

In addition, this makes it easier for StochasticNetwork, in that solution we will simply update the EV's station_id parameter before plugging it in.

This will break the plugin() method's API, however this is only really used internally, so it shouldn't be too much of a problem. For the next release we could log a warning and update the EV's station_id if plugin() is called with a station_id argument. Eventually we can just depreciate this argument.

Issue

Currently we have unit tests for individual parts of the simulator, but no integration tests to ensure different components of the simulator interact correctly

Ideas

• Use the tutorial as an integration test
• Write run method so that internal state can be readily accessed
• Automatic tests on github (eg TravisCI)
• Tests with known behavior (e.g. deterministic algorithm, specific event queues, etc.)

Is your feature request related to a problem? Please describe.
Saving simulations to json files is taking up too much room on my hard drive.

Describe the solution you'd like
I would like an option in to_json() to save as a gzipped file and an option in from_json() to load from a gzipped file.

We should improve the README so that users have a better idea of the capabilities of acnportal and how they can get started using it.

Issue

Currently, simulations that include a TOU tariff schedule as an input signal are inaccurate if they span longer than 1 month, as demand charge is calculated using the peak over the entire simulation instead of peaks during 1-month windows. We need to add support for longer simulations by including an accurate calculation of demand charge over several months.

Hi Zach,

I am trying to use pandas.read_json('...PATH\acndata_sessions.json') using the json file being generated by the website, however, I encounter an error in Python.

Can you help me fix this issue? I am trying to read json file for EV charging behavior.

Thanks!

Is your feature request related to a problem? Please describe.
As an acnportal user, I'd like to be able to keep track of the version of acnportal I am using so I can ensure reproducibility.

Describe the solution you'd like
acnportal should batch improvements into releases which should each be assigned a meaningful version number.

Describe alternatives you've considered
We could also add a version to any push to master. However, this could lead to lots of minor version increments. We should discuss the pros and cons to this approach.

Issue

Users should have the ability to initialize a charging network from a string with the site name rather than having to call a function to create the network. For example,

cn  = acnsim.sites.caltech_acn()

will initialize an instance of the Caltech ACN, but we'd like something like

cn  = acnsim.make_network(site='caltech')

to do the same. Additionally, we'd like to give users functionality to register new networks so they may be initialized in this manner.

Currently, only natively JSON-serializable signals can be serialized. We should write to/from_dict methods for existing signals.

I have stumbled upon a strange output from acnsim.
To be sure that it is not an issue with me messing around with code, I created a new virtual environment in python with the latest acnportal code from github. And the problem is still there

Description
Basically it seems like after a while (around 100 iterations) the simulation does not take into account the pilot signals to the EVSEs (the total aggregate current is always 0).

To Reproduce

import pytz
import numpy as np
from datetime import datetime
import matplotlib.pyplot as plt
from acnportal import acnsim
from acnportal import algorithms
import random

# Timezone of the ACN we are using.
timezone = pytz.timezone('America/Los_Angeles')

# Start and End times are used when collecting data.
start = timezone.localize(datetime(2020, 10, 5))
#end = timezone.localize(datetime(2018, 9, 6))

# How long each time discrete time interval in the simulation should be.

#network.
voltage = 208  # volts
aggregate_cap=150 #kW
evse_names=["EVSE-001"]#,"EVSE-002"]
#which implies
cap_current=aggregate_cap*1000/voltage

cn = acnsim.network.sites.simple_acn(evse_names,voltage=voltage, ...
aggregate_cap=aggregate_cap)

#the plugin events function is taken from from gym-acnsim
#it is crucial that it lasts enough iterations
iterations=500

def random_plugin(num, time_limit, evse, laxity_ratio=1/2,
                  max_rate=32, voltage=voltage, period=1):
    """ Returns a list of num random plugin events occurring anytime from time 0
    to time_limit. Each plugin has a random arrival and departure under the time
    limit, and a satisfiable requested energy assuming no other cars plugged in.
    Each EV has initial laxity equal to half the staying duration unless
    otherwise specified.
    
    The plugins occur for a single EVSE, whose maximal rate and voltage are
    assumed to be 32 A and  208 V, respectively, unless otherwise specified.
    
    Args: TODO
    Return: TODO
    """
    out_event_lst = [None] * num
    times = []
    i = 0
    while i < 2 * num:
        rnum = random.randint(0, time_limit)
        if rnum not in times:
            times.append(rnum)
            i += 1
    times = sorted(times)
    battery = acnsim.models.Battery(100, 0, 100)
    for i in range(num):
        arrival_time = times[2*i]
        departure_time = times[2*i+1]
        requested_energy = (
            (departure_time - arrival_time) / (60 / period) 
            * max_rate * voltage / (1 / laxity_ratio)
        )
        ev = acnsim.models.EV(arrival_time, departure_time, requested_energy,
                       evse, f'rs-{evse}-{i}', battery)
        out_event_lst[i] = acnsim.events.PluginEvent(arrival_time, ev)
    return out_event_lst

event_list = []
for station_id in cn.station_ids:
    event_list.extend(random_plugin(100, iterations, station_id))
event_queue = acnsim.events.EventQueue(event_list)
events = acnsim.events.EventQueue(event_list)

#print the pluginevents time
for i in range(len(events._queue)):
    print(events._queue[i][0])

sch = algorithms.UncontrolledCharging()
sim = acnsim.Simulator(cn, sch, events, start=start, period=period, verbose=True)
sim.run()

agg_current = acnsim.aggregate_current(sim)
plt.plot(agg_current)
plt.xlabel('Time (periods)')
plt.ylabel('Current (A)')
plt.title('Total Aggregate Current')
plt.show()

agg_power = acnsim.aggregate_power(sim)
plt.plot(agg_power)
plt.xlabel('Time (periods)')
plt.ylabel('Current (A)')
plt.title('Total Aggregate Power')
plt.show()

Expected behavior
I expect total aggregate current (and power) to be roughly evenly distributed across time.

Screenshots
Plots for the run:

System Info

• OS: Windows 10 Pro 20H2
• acnportal version: v.0.3.2
• run on jupyter notebook (v6.4.3)

Additional Info
This are the iterations of plugin events for the reported run:
5
8
11
14
17
19
24
30
38
44
52
57
59
63
65
69
71
75
79
83
86
90
93
98
100
105
115
119
127
132
136
140
144
156
162
170
174
180
183
189
193
197
201
205
218
223
229
239
242
249
254
262
266
268
274
276
279
287
295
299
305
310
313
317
319
321
328
332
334
336
343
351
354
358
362
366
371
378
383
390
393
401
405
419
424
428
431
433
435
437
439
445
449
454
462
466
476
488
491
493

Could you provide an example of using the simulator sim in step mode i.e.
sim.step(new_schedule)
as opposed to
sim.run()
I can't get the step function working.

The Caltech Webpage seems to be down since at least yesterday. API calls are not possible.

Issue

ConstraintSet is currently convoluted and hard to understand. We should explore how to make it simpler and more efficient.

Ideas

• Use a matrix representation to relate charging rates of individual EVSEs to aggregate currents within the network.
  • Pros
    • Computationally efficient
  • Cons
    • Could be difficult to construct for some networks. (We could build tools to help with this)

See the test_current_unbalance_nema test in the test_integration.py file for the actual test that raises a warning. We should look into where and why an invalid value is encountered, and fix this or at least provide a more technical description of why the issue occurs.

Hi Zach,

I was trying to run the first tutorial after installing the packages. But I encountered with the following error for this line.(from acnportal import acnsim)

ImportError: cannot import name 'stringify_path' from 'pandas.io.common' (C:\Users\jubai\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\io\common.py)

# Your code here, if applicable

Is your feature request related to a problem? Please describe.
We currently use TravisCI for automated testing. However this service can be very slow. For faster feedback we should move to Github Actions.

Describe the solution you'd like
Set up automated testing using Github actions.

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of acnportal.

Describe the bug
acnportal throws an error on import for pandas >1.2

To Reproduce
Import acnportal with pandas >1.2 installed.

Expected behavior
No error should be thrown.

Screenshots

======================================================================
246ERROR: acnportal.acnsim (unittest.loader._FailedTest)
247----------------------------------------------------------------------
248ImportError: Failed to import test module: acnportal.acnsim
249Traceback (most recent call last):
250  File "/opt/python/3.7.1/lib/python3.7/unittest/loader.py", line 468, in _find_test_path
251    package = self._get_module_from_name(name)
252  File "/opt/python/3.7.1/lib/python3.7/unittest/loader.py", line 375, in _get_module_from_name
253    __import__(name)
254  File "/home/travis/build/zach401/acnportal/acnportal/acnsim/__init__.py", line 1, in <module>
255    from .simulator import Simulator
256  File "/home/travis/build/zach401/acnportal/acnportal/acnsim/simulator.py", line 11, in <module>
257    from .events import *
258  File "/home/travis/build/zach401/acnportal/acnportal/acnsim/events/__init__.py", line 1, in <module>
259    from .event import Event
260  File "/home/travis/build/zach401/acnportal/acnportal/acnsim/events/event.py", line 7, in <module>
261    from ..base import BaseSimObj
262  File "/home/travis/build/zach401/acnportal/acnportal/acnsim/base.py", line 17, in <module>
263    from pandas.io.common import stringify_path, get_handle, get_filepath_or_buffer
264ImportError: cannot import name 'get_filepath_or_buffer' from 'pandas.io.common' (/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/pandas/io/common.py)

Is your feature request related to a problem? Please describe.

• I would like to be able to assign EVs to an open parking space dynamically when the EV arrives.
• When an EV arrives and no space is assigned, I would like the EV to be placed into a queue.
• When an EV leaves, an EV should be removed from the head of the queue and placed in that space.

Describe the solution you'd like

• Create a subclass of ChargingNetwork with the above features.

Issue

Currently, TOU Tariffs are specified in json files. CSV files are easier to edit and are more human-readable, so we should also support this format as an input.

Proposal

We will include a function in the tariffs module that converts a CSV file with a TOU tariff schedule into a JSON file with the appropriate format. Then, this new JSON file is read and processed by the simulator as before.

Feature

ACN-Sim should have support to EASILY calculate the cost of charging including TOU rates, demand charge, and tiered pricing.

Issue

Currently, charging_network throws an error if a constraint is added that constrains EVSEs that aren't in the network. There may be a use case in which we want to add constraints but only test on a subset of EVSEs of that network. If this is done by only registering this subset of EVSEs, an error will currently be thrown when the constraints are added.

Possible Solution

One possible solution is to register each missing EVSE when a constraint is added. These missing EVSEs are registered as None until the user explicitly registers them. All network accessors return parameters for only those EVSEs that are registered, but the internal EVSE list in the network stores all EVSEs.

Issue

max_recomp is currently an argument to the simulator. However, this is really a property of a specific scheduling algorithm, so it should be stored there.

Issue:
It can be frustrating when you don't know how far along you are in a simulation. Adding a simple progress bar with some timing stats about the simulation, i.e. percent of time steps remaining, percent of events remaining, would be helpful.

Issue

It's sometimes useful to be able to load an entire simulation from a file, but the only current way to do this is through pickle, which breaks if packages are updated and which is not secure. Writing our own serialization methods will allow us to send and load completed or partially-run simulations securely.

Is your feature request related to a problem? Please describe.
In the current implementation, ChargingNetwork.unplug() takes only station_id as an argument and unplugs whatever EV is connected to that station. However, this poses a problem if the EV is not actually at that EVSE, for example if we all the EV to swap with another EV after it is finished charging.

Describe the solution you'd like
The easiest approach is to include session_id as an argument (it is already included in the UnplugEvent), and just check if that session_id matches the session_id of the EV connected to the station.

This is not the most elegant solution, but it prevents us needing to invalidate previously serialized simulations.

Describe alternatives you've considered
My preferred approach is that instead of taking in a station_id string, unplug() should take in an EV object. It can then correctly find where that EV is plugged in and unplug it as needed.

The main challenge for this is that we are updating what is stored in an UnplugEvent which would cause problems if we want to deserialize from a previous version. Because of this potentially serious issue, I recommend the first option.

Is your feature request related to a problem? Please describe.
While ACN-Sim currently supports trace based simulation from ACN-Data, it should also support generating events from a stochastic model, i.e. Gaussian Mixture Model as proposed in ACN-Data: Analysis and Application of an Open EV Charging Dataset.

Describe the solution you'd like

Implement a StochasticSessionModel class to describe the stochastic model.
This base class can be extended for additional models such as GaussianMixtureModels, etc.

class StochasticEvents:
    """ Base class for generating events from a stochastic model. """
    def fit(self, data: List[Dict[str, Any]]) -> None:
        """ Fit StochasticEvents model to data from ACN-Data.

        Args:
            data (List[Dict[str, Any]]): List of session dictionaries. See DataClient.get_sessions().

        Returns:
            None
        """
        pass

    def sample(self, n_samples: int) -> np.ndarray:
        """ Generate random samples from the fitted model.

        Args:
            n_samples (int): Number of samples to generate.

        Returns:
            np.ndarray: shape (n_samples, 3), randomly generated samples. Column 1 is the arrival time in hours
                since midnight, column 2 is the sojourn time in hours, and column 3 is the energy demand in kWh.
        """
        pass

    def generate_events(self, sessions_per_day: List[int], period: float, voltage: float, max_battery_power: float,
                        max_len: int = None, battery_params: Dict[str, Any] = None, force_feasible: bool = False
                        ) -> EventQueue:
        """ Return EventQueue from random generated samples.

            Args:
                sessions_per_day (List[int]): Number of sessions to sample for each day of the simulation.
                period (int): Length of each time interval. (minutes)
                voltage (float): Voltage of the network.
                max_battery_power (float): Default maximum charging power for batteries.
                max_len (int): Maximum length of a session. (periods) Default None.
                battery_params (Dict[str, object]): Dictionary containing parameters for the EV's battery. Three keys are
                    supported. If none, Battery type is used with default configuration. Default None.
                    - 'type' maps to a Battery-like class. (required)
                    - 'capacity_fn' maps to a function which takes in the the energy delivered to the car, the length of the
                        session, the period of the simulation, and the voltage of the system. It should return a tuple with
                        the capacity of the battery and the initial charge of the battery both in A*periods.
                    - 'kwargs' maps to a dictionary of keyword arguments which will be based to the Battery constructor.
                force_feasible (bool): If True, the requested_energy of each session will be reduced if it exceeds the amount
                    of energy which could be delivered at maximum rate during the duration of the charging session.
                    Default False.
            Returns:
                EventQueue: Queue of plugin events for the samples charging sessions.
        """
        pass

Is your feature request related to a problem? Please describe.
Currently algorithms within ACN-Sim use the real departure of the EV. However, this value is not always known perfectly, so instead we should use estimated_departure which can be provided based on real data taken from the user OR predicted in some other way.

Describe the solution you'd like
Replace departure with estimated_departure in ACN-Sim algorithms.

Issue

Integration tests only test tutorial 2; we should test a wider variety of situations, especially using the new networks. Also, some functions remain untested. We should get a code coverage app and make sure all the code is tested.

Currently, the serialization feature of the simulator only records the versions of acnportal, numpy, and pandas used. To record the environment, a pip freeze would be more adequate, along with an acnportal git hash if available.