Homec4science

Provide a workflow to restart Harbormaster builds

Authored by Mike Riley <mikeriley@yelirekim.com> on Sep 2 2016, 05:22.

Description

Provide a workflow to restart Harbormaster builds

Summary:
Ref T10867 for original use case. This workflow provides a plausible way for administrators to stop the daemons when performing upgrades or maintenance, then bring those daemons back up without resulting in the failure of builds that were running at the time.

On our organization's phab install, builds are running 24/7. The majority of these builds last for at least several minutes, and contain build steps which fail if interrupted and then resumed, as happens when turning daemons on and off.

Instead of allowing these build steps to resume execution as normal, this workflow will instruct active builds to restart their entire build process instead of just resuming whichever step they were on.

Test Plan:
contrived a build plan which would fail if resumed partway through:

  • lease a working copy
  • command touch restart_{build.id}
  • command test -e restart_{build.id} && rm restart_{build.id} && sleep 60

followed old procedure:

  • run a few of these builds manually
  • ./bin/phd stop
  • ./bin/phd start
  • saw the builds fail

followed new procedure:

  • run a few of these builds manually
  • ./bin/phd stop
  • ./bin/harbormaster restart --active
  • ./bin/phd start
  • saw the builds pass

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T10867

Differential Revision: https://secure.phabricator.com/D16485

Details

Committed
Mike Riley <mikeriley@yelirekim.com>Sep 2 2016, 15:32
Pushed
aubortJan 31 2017, 17:16
Parents
rPH0eb5a80e7b68: Fix typo at calendar transaction
Branches
Unknown
Tags
Unknown

Event Timeline

Mike Riley <mikeriley@yelirekim.com> committed rPH403073c989b4: Provide a workflow to restart Harbormaster builds (authored by Mike Riley <mikeriley@yelirekim.com>).Sep 2 2016, 15:32