Comments (15)
Fenzo supports scalar resources but currently not set resources.
That said, the constraint evaluator is called with list of tasks already running on the host from before as well as any new assignments already performed in the current scheduler run. So, if first task1 was assigned, then when constraintEvaluator is called for task2, it would contain task1 in the callback. ConstraintEvaluator.get VirtualMachineCurrentState() will return VirtualMachineCurrentState object which has the method getTasksCurrentlyAssigned() for this. See the JavaDocs for details.
from fenzo.
I thought the VirtualMachineCurrentState object inside the ConstraintEvaluator.evaluate() method contained the state of the cluster itself (i.e. stuff that is managed by taskScheduler.getTaskAssigner and taskScheduler.getTaskUnAssigner), but it looks like VirtualMachineCurrentState contains scheduling assignments in the current taskScheduler.scheduleOnce() call?
In any case, the TaskRequest I implemented isn't going to hardcode "foo-a" in the ConstraintEvaluator, but "foo-*". Just wondering, how do I separately keep track of "foo-a" and "foo-b" being claimed? Will mutating the TaskRequest provided in ConstraintEvaluator.evaluate() work (or does Fenzo create new copies of the TaskRequests internally?
from fenzo.
VirtualMachineCurrentState. getTasksCurrentlyAssigned() gets you tasks that are currently assigned in the current scheduler iteration. Where as, VirtualMachineCurrentState. getRunningTasks() gets you tasks that are already assigned from before (that's what you refer to as the cluster state, managed via taskAssigner and taskUnassigner).
So, you have access to both.
You can mutate TaskRequest, there are no copies created, Fenzo only holds a reference to the objects you create. Since there is no direct support for set type resource right now, maintaining some state in your assigned tasks for which of the set values they pick could help.
Feel free to open a new dev issue to support resource sets.
from fenzo.
Oh, ok; thanks for the info!
from fenzo.
I tried to implement a constraint:
evaluate(TaskRequest taskRequest, VirtualMachineCurrentState targetVm, TaskTrackerState taskTrackerState)
and inside the evaluate call, I attempt to lookup all the SetResources in an offer from the VM as follows:
targetVm.getCurrAvailableResources().getOffer()
I create a TaskRequest with this constraint evaluator, and submit this TaskRequest as well as one VMLeaseObject into scheduleOnce(). However, getOffer() line of code returns a null pointer. Why is this the case? I note that I had called scheduleOnce() with a non-null list of VMLeaseObjects...
I also note that targetVm.getCurrAvailableResources() returns a com.netflix.fenzo.AssignableVirtualMachine, not a VirtualMachineLease. I looked into the code, and the only reference to an AssignableVirtualMachine is assignableVMs.getDummyVM() in TaskScheduler.java. Is it the case that scheduleOnce() initializes a dummy VirtualMachineLease, which will be populated as schedulerOnce() iterates through the task requests, which would explain why getOffer is null?
(I am using fenzo 0.10.0 as published in maven)
from fenzo.
Line 809 of AssignableVirtualMachine.java:
private VirtualMachineCurrentState vmCurrentState()
returns a VirtualMachineCurrentState to ConstraintEvaluator.evaluate() in line 626 and 843.
However, this VirtualMachineCurrentState is a totally new object with no actual reference to a Protos.Offer.
How do I access the Protos.Offer from inside ConstraintEvaluator.evaluate()?
from fenzo.
If an agent has multiple offers, they are combined for resource assignment purposes. Therefore, the getOffer() implementation for the targetVm's current available resources returns null since there isn't necessarily one offer.
We can, however, enhance this to provide a list of offers on the targetVM, which will help your case.
Let me know if that works for you.
from fenzo.
Yes, that works. Either a list of offers on the targetVM, or a materialized view/offer that is the sum of the current offers available for the targetVM would be good, though I would prefer the latter since it will be easier to work with as a client of Fenzo.
from fenzo.
Yes that works. Either a list of offers or a materialized view/offer that represents all current available resources the targetVM has, not including the ones that have been claimed by earlier taskRequests in the current scheduling assignment run. I prefer the materialized view, because it is easier to work with as a client of Fenzo.
How soon do you expect this feature to land in?
from fenzo.
The materialized view would not be ideal since there may be resources in the offers (such as persistent volumes) that Fenzo doesn't yet directly support and therefore will not be able to accumulate correctly for a materialized view. Making a collection of offers available can be done easily. I can do that early next week if that works.
from fenzo.
Sounds good
from fenzo.
Fenzo 0.10.1 is ready for use and makes the Mesos resource offers available to constraint evaluators.
from fenzo.
Thanks! And to be clear, getAllCurrentOffers() does NOT reflect the current status of the offers from previous task assignments in the same scheduling loop right? For example, if I run scheduleOnce on a list of two tasks, and task1 claims X, then X will still be visible to task2 through getAllCurrentOffers(), right?
from fenzo.
Strictly speaking, an individual offer of a host is not claimed by a task. Rather, cumulative resources from multiple offers of a host are assigned to multiple tasks.
So, effectively, your constraint evaluator will see offer X for both invocations - once for task1 and again for task 2.
from fenzo.
I am closing this based on the discussion above. Feel free to open it if needed or open a new issue. Thank you.
from fenzo.
Related Issues (20)
- contributor agreement. HOT 8
- NamedResourceSetRequest HOT 2
- Resource ranges and "greedy" scheduling HOT 4
- Support v1 Protos (or provide utility to convert) HOT 5
- Support dynamic fitness threshold HOT 3
- Allow for custom shortfall evaluators HOT 4
- Execution failed for task ':fenzo-core:compileJava'.> 无效的源发行版: 1.8 HOT 2
- Update Mesos dependency
- BinPacking with weights HOT 2
- Incorrect handling of reserved resources HOT 2
- Unsafe concurrent access to unknown lease collection in AssignableVMs class
- Pluggable ENI fitness evaluator HOT 1
- After reading Fenzo, I don't understand how to get Framework information on Mesos?
- After reading Fenzo, I don't understand how to get Framework information on Mesos?
- DisableVM does not remember hosts HOT 2
- .travis.yml: The 'sudo' tag is now deprecated in Travis CI
- Dynamic ResAllocs
- CVE-2018-7489 on jackson-databind 2.4.5 HOT 1
- Unused variable
- Unreleased Resource: Streams HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fenzo.