worker: run repo maintenance during idle time (Bug 2037216) #1135 (lando@main)
| Warnings |
|
|---|---|
| Blockers |
|
| Branches | idle-strip -> main |
| Repo | lando@main (git) (git://github.com/cgsheeh/lando.git) |
| Author | cgsheeh |
| State | open |
| Commit Title | worker: run repo maintenance during idle time (Bug 2037216) |
| Commit Body | Lando's workers typically run repo-cleaning commands at the beginning of each job processing time. In hg workers, we run `hg strip` at the start of each job to remove previously-created stale commits, despite those commits not interfering with the job completion. In Git, we take the opposite approach and simply ignore the temporary work branch - it is never cleaned up. Add a new "repo maintenance" step that runs while the worker is being throttled due to no jobs remaining in the queue. To avoid running excessive maintenance, the runtime of the last maintenance run for each repo is recorded in the worker, and maintenance is skipped if it has been completed within the threshold. For Mercurial, move the `hg strip` command into this maintenance task, which should save us about 8s for each push to try. For Git, add a cleanup of the stale working branches, so we no longer have thousands of temp branches in our worker repos. After this change, each `HgSCM.clean_repo` call sites always pass `strip_non_public_commits=False`, while `GitSCM.clean_repo` call sites always pass `True`. Remove the kwarg and make each behaviour the default. |
Landings
Not yet Landed
There has been no attempt to land revisions in this stack.