Provide a workflow to restart Harbormaster builds
Summary:
Ref T10867 for original use case. This workflow provides a plausible way for administrators to stop the daemons when performing upgrades or maintenance, then bring those daemons back up without resulting in the failure of builds that were running at the time.
On our organization's phab install, builds are running 24/7. The majority of these builds last for at least several minutes, and contain build steps which fail if interrupted and then resumed, as happens when turning daemons on and off.
Instead of allowing these build steps to resume execution as normal, this workflow will instruct active builds to restart their entire build process instead of just resuming whichever step they were on.
Test Plan:
contrived a build plan which would fail if resumed partway through:
- lease a working copy
- command touch restart_{build.id}
- command test -e restart_{build.id} && rm restart_{build.id} && sleep 60
followed old procedure:
- run a few of these builds manually
- ./bin/phd stop
- ./bin/phd start
- saw the builds fail
followed new procedure:
- run a few of these builds manually
- ./bin/phd stop
- ./bin/harbormaster restart --active
- ./bin/phd start
- saw the builds pass
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, epriestley
Maniphest Tasks: T10867
Differential Revision: https://secure.phabricator.com/D16485