In our research on cluster scheduling, we learnt two things:
Firmament is our answer to these challenges. It offers:
Using a min-cost optimisation over a graph, Firmament finds the best placement for each task or container in accordance with the scheduling policy.
By customizing the underlying graph and setting the costs on its edges via a callback interface, users can customize Firmament to apply their own policies.
Firmament's incremental min-cost, max-flow solver makes rapid, sub-second scheduling decisions even at Google scale (12k machines).
... and best of all, it's all open-source:Get started!
Firmament is based on a graph-theoretic construct called a flow network, over which it runs a minimum-cost optimization. Unlike heuristic-driven schedulers, Firmament's approach is guaranteed to find the policy-optimal assignment of work to cluster resources.
The below visualization illustrates the principle using a
You can use the buttons to manipulate the cluster – keep clicking until they're grayed out!
There are several existing cluster managers, and we recognize that users want to use Firmament as a scheduler for their favourite cluster manager.
All of our code is open-source software under the Apache 2.0 license.Get it from GitHub
Please file an issue if you encounter problems, or send us a pull request if you make changes!
If you want to dive deeper, we have plenty of material for you!