sinanmohd / evanix Goto Github PK

Implement the greedy selection algorithm for the "counting" cost

The return value of the solver part of evanix should contain enough information to generate a report saying (0) how many derivations in total will be built (in future versions also how many compute hours we expect to expend), (1) which "targets" will be built, (2) which "targets" will remain unsatisfied. Optional features: print how much extra budget is required to satisfy one more target. Both --dry-run and the normal mode should be able to produce a report like that. The return value of the solver should probably be an immutable structure, and the "solver" should be a simple enough function to be used in a unit test

For the "counting cost" setting, use iterative approximation to the solution of the integer problem:

• https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.milp.html
• https://github.com/ERGO-Code/HiGHS/blob/0095d61a2e37df88cc8616b374350c5be260e862/examples/call_highs_from_c.c#L287

Note that DAG constraints correspond to inequalities

The midterm evaluation is starting now, let's decide what we want to include in the report

What's in:

• The greedy demo
  • #10 to make it more presentable?
  • #7 for the same reason?
• Literature overview

What's left for the 2nd half of the term:

• Dump hydra's build times and output sizes for a single evaluation
• Implement an "expected times" PoC using the same HIGHS solver
• Decide on what's a good schema for storing and distributing said hydra metadata; estimate how much the full hydra history would take to store then; decide on the schema for distributing evanix's statistical models
• Maybe talk to the infra people and dump all of hydra's history in said format
• Maybe reimplement evanix in rust/go

@sinanmohd wdyt

Running evanix on https://github.com/ggerganov/llama.cpp takes, in the --dry-run mode, about 20s, whereas running nix-eval-jobs takes about 6s. Per @sinanmohd's hypothesis, this is due to querying the substituters about the cached store paths too much (does nix not maintain a negative response cache for these queries?). The performance needs to be brought down to the same order of magnitude as nix-eval-jobs so that we can stop talking about the greedy PoC

Eventually we want to get Nix's scheduler entirely out of the picture:

• Start building from the leaves (rather than from "direct" targets). A "leave" in our context is a .drv for which no extra inputs have to be built.
• Create temporary roots for the dependencies and keep them until it has no unbuilt reverse dependencies left
• Cleanup (shouldn't be necessary unless evanix was interrupted)

(very low priority)

Instead of choosing between the whole eval to finish or disabling scheduling altogether (i.e. scheduling jobs one by one as they come), do something like like "wait X milliseconds to accumulate a batch of jobs, run the solver on this subset, schedule some builds, then wait another X milliseconds to extend the batch and re-run the solver to get a better plan"

It is OK to focus on the "complete eval first" scenario and implement batching later: our scope is to find a good way to choose a build plan, not in speeding up the eval