fullammo / flowbstertoil Goto Github PK

The FlowbsterToil platform is not tested yet. I've got to get used to the Angular Testing concepts besides my actual knowledge of unit testing overall. Gotta learn Jasmine framework for assertion, and programatically initialize angular components and provide the building steps for the angular application from code for testing purposes.

Once you designed a workflow with the FlowbsterToil platform, Users could use these 'templates' to build infrastructures and start the experiments on the target cloud with the help of the Occopus orchestrator.

The platform should have a tree data table view that contains these 3 levels of abstraction.
-WorkflowA (The used template for the built infrastructure)
-DeploymentA (The actual deployment, where you can decide in which cloud environment the infrastructure will be created)
-ExperimentA ( The different datasets how the built infrastructure will be utilized )

Before building an infrastructure with Occopus we are going to ask for some information about the Deployments context. Actually its going to consist of a :

• Name Input

• Selection For the actual deployment Type

• A Submit button to actually start the infrastructure

• When submitting the component should start building the infrastructure and save it to FireStore under the given entry.

All this wrapped in a Template layer dialog.

The Gather components information can be given at the drawing level of a workflow at the moment.

It would be nice to give it at the deployment or experiment level. It brings more flexibility to the user

At the moment we are unable to control the selection of records on the datatable and it breaks our multiple selection mixed with contextMenu.

1. (check #4118 PrimeNG issue for better UX)

• Change the name or the description while clicking on them on the fly. (?)

The little actions are irritating my eyes. So it would be nice to right click on a given record and utilize the already implemented action methods.
Right now PrimeNG is not supporting this kind of funcionality with the datatable. But somebody made a feature request about it.
To keep track primefaces/primeng#4118

A documentation should be added for users, how to interact with the platform and what you can achieve with it. The whole lifecycle of the Occopus and Flowbster tools should be covered in this tutorial.

Also there are some requirements for the Platform that are the following:

• Input node's properties correctly included in the infrastrcuture descriptor.
• Collector(ip,port) variable name changes to Gather port variable name's.
• "Update workflow entry" tooltip information update
• Workflow submission also informs when no changes made visually.
• Build Status information of the workflow and the additional IP addresses.
• Node creation should be changed to double click event instead of a single click event.
• Submit button name change to Save.

We have to provide a way to let the user decide (based on the Auth.data.yml file on the Occopus backend) , where he wants to deploy their infrastructure template.

In the infrastructure descriptor, for every node there is a filter parameter which can be set based on the resource data, where to deploy that actual node.

For this feature on the platform we have to make these preparations:

• Get Descriptor Service to be able to insert filters to the nodes and create a fine yaml, based on deployment context information: which can be anything in the resource list or unspecified.

We come to conclusion about that for the deployment layer #16 we would like to :

• Store the parent graph for every deployment datastructure for further manipulation.

Single record functions:

• Destroy a deployment on the Occopus side(That also means delete from the FireStore database)
• Inspect the actual status of the deployment on the paper of the infrastructure visually.
• Inspect the actual status in details of the deployment on a dialog.
• Status bar: Pending, Active, Destroyed
• A button to send the actual deployment again
• Delete button for actually deleting and destroying an infrastructure.
• [x ] Create Experiment modal

Multi operations:

• Destroy
• ReSend
• Delete&Destroy

Other functionalities:

• Get informations from the occopus messaging service about the actual infrastructure's process, and try to color the actual node's accoruding to the informations got from occopus.

• Toast messages from the actual node's occopus info in a sidebar.

• We got to know on this level which inputs are free. To list for the users the free inputs and let them decide where they want to send their data to.

• Create a dialog for starting an experiment

• Create a generic form from the free inputs

• File upload tab

• Dot and Cross combinations of the products.

The data is a given file, which can be found on the Occopus documentations as an example.

Once we sent the infrastructure description to Occopus it gives us an infrastructure ID back. That ID gotta be stored in our database under the templates collection to reference it right.

For the deployment model we need these informations:

• Name of the deployment
• The Deployments type that was used (sigma,ec2 etc).
• The status of the actual deployment (Pending,Active)
• ID of the delpoyment that is used on the occopus side
• Date of the creation
• ID of the deployment at our database side

Once we have these deployments the user should interact with it:

• Send different data sets to process by the deployment
• Expand to see the different experiments (DataSets) which will be processed.

This should be displayed in a separate angualar component.

We could provide a way to the user to populate a resource list based on #19 issue.

It could be efficient to keep track on the platform about any resource that we add to the Occopus utility as resource.

At first these things should be done:

• A service to provide database operations and hold resource data.
• Component for resource list manipulation.
• A simple list of resources is enough as datastructure on the FireStore database under the user.

We want to store every layers information in our FireStore database. We would like to do almost the same operations with them on every layer. but the path and some implementations changes over time. I think this could be overcome with one generic service implementation.

The platform should be able to destroy,build and get information from the running Occopus infrasttructure proccess. Try these functionalitys and their boundaries from the Occopus side.

The joint service is getting way big. It would be nice to sanction its functionalities into smaller cohesive groups, for better testing and understanding later.

Whenever you filter for something, and you get the matching records you may want to select every record in that filter. Thats right now not possible, because the header checkbox selects every item that is present in the datatable. Its not concentrated on the filtered items.

At the workflow infrastructure template layer the user may want to:

• Sorting For the name attribute.
• Filter by the name attribute.
• Get Global filtering to search descriptions too.
• Select entries with a checkbox column.
• A Column for expanding the template which contains the different deployments of the template.
• Use Pagination or scroll to avoid massive amount of templates
• Have a "sidebar" for multi and one time operations.
• Have an Action Column which includes operations for individual rows

These Actions should be implemented for every data row:

• Delete,
• Build an infrastructure from yaml
• Edit (maybe only the graph?)
• Copy
• Show the readonly template graph in drawing area by clicking on a a button

The action side bar should contain these functionalities:

• Add new Entry.
• Multi Delete
• Multi Build infrastructure from yaml (I think occopus is not ready yet)
• Multi Copy

It would be useful to add explanations to every function and class that is represented in the project to support the compodoc documentation generation tool.

To get additional infromation about our constructed infrastructure, on the Occopus side we have an opportunity to let the orchestrator send infromation to our client side application about the construction

• We need a data model in the fireBase database to store FCM tokens to build up the communication
• Send these FCM token informations via HTTP to Occopus
• Represent informations to the user via push notifications or toastr messages on the screen.
• Manipulate the deployment graph based on these informations.