Resource ranges and "greedy" scheduling about fenzo HOT 4 OPEN

These are certainly possible with some changes in Fenzo.

I see why approach 2 may be more optimal for you. Although, one aspect you may be giving up from approach 1 is getting a "more optimal" assignment if you were to wait for sometime before lowering the resource requirements. For example, a task requiring 8 CPUs, but, willing to accept just 4 CPUs might get assigned 4 CPUs right away in approach 2. Where as, waiting for, say, a few seconds may have gotten the more optimal assignment of 8 CPUs. Without too many changes, there is, of course, no way to tell if waiting will in fact get you a better assignment.

Also, there could be some challenges from approach 2 in incorporating the multiple task assignments (from permutations) for the same task, where an assignment may have to abort previous assignment for its variant, etc. Your concern on its feasibility is very valid.

There may be another approach here.

Approach 3:

1. Define a new task request type in Fenzo, something like VariableSizeQueuableTaskRequest. This task request has a list/array of scalar requests for entities like CPUs, memory, etc.
2. Fenzo would then try to satisfy the request starting with the maximum of the requested quantity and iteratively down to the minimum, stopping with the max value it could allocate. The fitness value would be a function of where in the list it was able to assign
3. The task assignment result would indicate the actual quantities assigned

This is in some ways a variation of your approach 2 but better supported in Fenzo itself instead of your framework having to create multiple "duplicate" tasks for the same task. It then removes the complexity of having to abort previous assignments, concurrent assignments of same offers, etc.

This would require changes in Fenzo, but, my preliminary thinking is that it can be done in a way such that other Fenzo users not using this feature would be unaffected.

This requires further thought. I haven't considered it in full detail yet. We may have to iterate on the details.

from fenzo.

I appreciate the quick feedback. We don't need to pick a solution for a few weeks so there's time to iterate. I mostly was interested up front in your level of interest and whether or not our approaches were way off! We look forward to more discussions.

from fenzo.

Just following up to see if you had any more insights on this scheduling approach.
My current thought is that my "Approach 2" is too complex, but that I could implement "Approach 1" fairly easily... just to get the capability available in our scheduler.
Then, maybe down the road a bit, if "Approach 3" were to become available in Fenzo, I would just migrate to that.

Any thoughts?
Am I over-simplifying "Approach 1?"
Do you think that "Approach 3" could happen one day in Fenzo?
And one final, related question... I have some interest in poking around the Fenzo codebase. If I could get something working (with appropriate unit tests and documentation, of course) would you be willing to work with me on a pull request?

Thanks again!

from fenzo.

Your "Approach 1" sounds reasonable. This may work well for a lot of cases until the scale and complexity of the use case increases.

It is too early to say when/if Approach 3 will be available in Fenzo. This will need to be driven by folks with the most need for it.

And one final, related question... I have some interest in poking around the Fenzo codebase. If I could get something working (with appropriate unit tests and documentation, of course) would you be willing to work with me on a pull request?

Sure. Are you referring to "Approach 3" or a simpler one?

from fenzo.