Support daemon autoscaling in libphutil
Summary:
Ref T7352. Autoscaling allows daemons (particularly, the taskmaster daemon) to scale up (launch more copies) or down (have some copies exit and not restart).
The primary goal is to improve resource utilization in the cluster. In particular:
- Instances with relatively little activity will automatically scale down to 1 taskmaster.
- Instances with a lot of activity can scale up to some higher-than-average ceiling.
In the cluster, compared to the current strategy of a fixed number of taskmasters per isntance, this uses less resources in all cases except when the daemons are fully loaded (when it uses the same amount of resources), and allows us to allocate resources much more efficiently.
This doesn't really impact normal installs much. They'll still get autoscaling, but the memory footprint of daemons isn't large enough to matter in most cases.
This (and the next diff) don't actually autoscale taskmasters yet, they just build all the autoscaling infrastructure.
Test Plan:
- Ran phd debug --autoscale intensity.
- Saw pool scale up to 8 daemons on odd minutes, launching one new daemon every few seconds, then stabilize.
- Saw the pool scale down to 1 daemon at 15 seconds past the minute on even minutes. The pool would normally scale less abruptly, but the busy-ness function for the HighIntensity daemon means this is the expected behavior.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7352
Differential Revision: https://secure.phabricator.com/D11859