aileenproject / aileen-core Goto Github PK

When running an aileen- use case on an Aileen box, one should not have to downlaod aileen-core manually. It should be a pip dependency.

The issue to solve is that we need to run manage.py. Either we

• make a function, which aileen-core's setup.py installs as an entry point: https://lincolnloop.com/blog/goodbye-managepy/ Better even: Make two, one only for run_box, one for stop_box. I did not succeed with this due to some path problems, but it seems possible. Maybe we aren't using the best way to import code. The Django coding style says to use "relative imports for local components"
• Make a script which will go to the installation directory (https://stackoverflow.com/questions/122327/how-do-i-find-the-location-of-my-python-site-packages-directory) and then call manage.py from there. I guess we'll have to make sure an .env file will be read in before we change to that directory. Also we should change back when that script ends. This solution feels a little more inconvenient.

Right now the server allows to look at data through a standard UI and to download the KPIs via a dedicated endpoint. This is good to have.

However, to make custom processing or custom frontend apps possible, we should offer all uploaded data (aggregations and events if available) via a API. Probably Django-REST.

Right now (due to historical reasons), each aileen-core box uses one specific kind of aggregation: Count the distinct observables which were seen, per hour or per day.

We can allow much more, like max/mean/min value, and we can simply aggregate across all observables or add this grouping-by-observable step (which we do now).

We'll have to add a model class Aggregations, so the box knows what to do. For data quality, right now it seems that the server is the best place to configure them centrally, and boxes simply download them.

If there is more than one aggregation (e.g. the user wants min as well as max), the box uploads twice the amount of aggregations.

For the UI, the server will allow the user to select which aggregation needs to be shown. The API (see #9 ) will of course also reflect this.

If this is false (default is true), then skip a few tmux sessions in run_box, which upload data.

A new setting DELETE_EVENTS_AFTER_HOURS (default -1) should specify after how many hours an event should be deleted form the box.

We'd need another tmux window which checks every hour if there are older events and deletes them. Or this can happen in a more general window which runs every hour anyways.