realvnf / bjointsp-adapter Goto Github PK

Discussion on how to best evaluate B-JointSP and compare it against other algorithms (mostly DDPG). Related to https://github.com/RealVNF/evaluation/issues/29

Current approach:

• Fix run duration to 6 (instead of 100 like DDPG) and pass info from all incoming flows within the 6 timesteps as input to B-JointSP
• Reason: B-JointSP treats all specified flows as long-running parallel flows competing for resources.
• But flows arriving more than 6 time steps after another are sequential and do not compete for resources. If we gave all these flows as input B-JointSP would over-estimate the resource requirements, conclude that there are not enough resources, and place all VNFs uniformly to minimize-over subscription
• Resulting placements and mappings of flows are then converted to placments and scheduling rules that are passed to the simulator
• Another issue with runs of length 6:
  • If no flows arrive during a run, B-JointSP is not even called, but the run is simply skipped without changing the schedule
  • But with random arrival if flows arrive for one ingress but not for another, B-JointSP removes all VNFs for the unused ingress. The next following flows will then be dropped because no VNFs are available for the ingress.

Alternative:

• Goal: Use same run duration for both DDPG and B-JointSP. DDPG doesn't work well with run duration 6.(?) So let's set run duration 100 for both
• Calculate the average data rate during the run in the simulator and provide it to the B-JointSP adapter. Eg, 10 flows of size 1 within 100 time steps (det. arrival) --> Rate of 10/100 = 0.1
• Take the processing time (VNF delay + flow length) into account, which affects the required resources (in our case 6). --> 0.1 * 6 = 0.6. Round up to 1.
• Divide by flow size (1) to calculate the number of flows. Here: 1/1=1 (but may be more during bursts or MMP).
• Thus pass 1 flow of size 1 to the B-JointSP

This way, we would calculate roughly the data rate that needs to be served in parallel. But still split it into separate flows of the pre-defined size (here, 1) such that B-JointSP can embed the flows separately.

Does that make sense? @adnan904 @qarawlus

As discussed:

• Always keep prev_schedule from last run.
• Before the first run, init prev_schedule to uniform distribution for all nodes
• When B-JointSP is called and flows arrive for some source but not for all, currently it normalizes 0, 0, 0, ... setting a uniform distribution for nodes without flows. Then when new flows arrive, they are dropped because they are sent to nodes without VNFs due to the uniform distribution.
• Instead, if B-JointSP is called and there are no flows for a sources, keep its prev_schedule rather than normalizing the 0s and resetting to a uniform schedule

Please add an overview in the Readme (before setup) that contains info about what the adapter does. At least:

• How do simulator observations and inputs of B-JointSP differ? How does the adapter convert from simulator observations to inputs for B-JointSP?
• How do B-JointSP outputs and actions/inputs to the simulator differ? How does the adapter convert them?
• Any other assumptions we made in the adapter? Eg, assume only one SFC (I saw a comment in the code).

Best to check comments in the code, old notes/issues, eg, CN-UPB/B-JointSP#17, if you're not sure anymore. A few bullet points are fine, no long text please.

So that we can run multiple experiments in parallel. And don't have to write to file the whole time. Shouldn't break anything of the existing functionality.

Not sure whether to do this in the adapter or B-JointSP itself. Probably both.