TaskRequest should support wait queue expiration about fenzo HOT 8 CLOSED

Fenzo does not store task requests internally, it only stores the leases (aka, offers). There are already several ways to ensure the leases are expired timely.

Or, are you suggesting that you'd like to have a queuing mechanism in Fenzo?

from fenzo.

I have a framework with Fenzo running in a infinite loop on another thread (this is based on the example framework code provided in Fenzo repo). What I noticed was that when I insert a TaskRequest and resources together in scheduler.scheduleOnce(), and the TaskRequest has a resource requirement that does not exist yet in the Mesos cluster, then I get a SchedulingResult where the VmAssignmentResult has no TaskAssignmentResultt - this is expected. However, after some time, when I register a new Mesos agent that offers resources matching the task's resource requirements, I notice that Fenzo will now output a TaskAssignmentResult that pairs that TaskRequest and the new resource together - this is good behavior. I did not re-add the original task in via scheduler.scheduleOnce(), so I concluded that Fenzo must have some internal data structure that holds on to the TaskRequest that was sent in by the previous scheduler.scheduleOnce() call.

What I would like is for Fenzo to not output a TaskAssignmentResult if the TaskRequest was sent in by a scheduler.scheduleOnce() call over N minutes ago, and for that TaskRequest to be kicked out from Fenzo if it is considered too old. In other words "Please try to make a VM assignment for this TaskRequest but give up after N seconds and let me know about this". Is this currently possible?

from fenzo.

I was traveling with limited connectivity, so my response is delayed.

Although Fenzo is battle tested, the sample framework is not, it is provided as an example for how to use Fenzo in a framework. I would not recommend using the sample framework itself for any mission critical applications. Read the comments provided in the sample for more details.

Since Fenzo currently does not support queuing of tasks, you will have to support queuing in your framework. The sample framework is a simplistic example that maintains the collection of pending tasks to call Fenzo repeatedly.

from fenzo.

Ok, I see. When I call scheduler.scheduleOnce() with a list of tasks, is it possible for me to bias the scheduling so that older-submitted tasks get prioritized first if they can be assigned to resources (i.e. if two task requests are given and score equally well in terms of fitness, but task request A is older than B, then choose A)? Which FitnessCalculator or ConstraintEvaluator would be good for this?

from fenzo.

Sure, Fenzo processes tasks in the order it receives via the List of task requests in the scheduleOnce() call. So, if you order the older tasks ahead in the list, they will be considered for resource assignment ahead of newer tasks.

The fitness calculator is to choose an agent, for a given task, from multiple agents that can fit the task at the moment.

from fenzo.

@q10 You may want to look at the recent introduced tasks queues in Fenzo. See details here

from fenzo.

Thanks

from fenzo.

Feel free to reopen this or a new issue if you need further help.

from fenzo.