Race condition in target branch selection in the ABI-API-checking job

[gilles-peskine-arm]

The ABI-API-checking build in the pr-merge job runs abi_check.py between the current state of the target branch, and a merge of that with the PR branch. At least that's the theory. More precisely, what happens is:

1. checkout_repo.checkout_repo() calls checkout scm, which is a Jenkins plugin that performs a merge of the branch under test with the target branch, where “branch under test” and “target branch” are conveyed via automation related to the job type that pr-merge is built on.
2. Our code calls git fetch origin ${CHANGE_TARGET} to retrieve the state of the target branch.

There is a race condition between (1) and (2): if the target branch is updated in the meantime, the merge of the branch under test gets compared with a newer version of the target branch. If the newer version has API extensions, the API check will report them as being removed. If the newer version has ABI changes, the ABI check will report them in the other direction.

Example: this happened in Mbed-TLS/mbedtls#5524 (https://jenkins-mbedtls.oss.arm.com/blue/organizations/jenkins/mbed-tls-pr-merge/detail/PR-5524-merge/13/pipeline/), for example.

Replaying an old job with the “replay” button won't help because the commits used by step 1 are cached. If you do this, you can only widen the race condition.

This doesn't happen often, so if it's hard to fix, we may want to just document it. Meaning both remembering that this happens, and documenting a replay procedure that refreshes the choice of git commits.

[mpg]

documenting a replay procedure that refreshes the choice of git commits.

Is there another way of doing that than pushing again (possible a zero-diff force-push)?

[gilles-peskine-arm]

You can run the job again with the same parameters: that's different from a replay. For pr-merge, this means going to the pr-merge job, selecting the “Pull Requests” tab, scrolling down to the desired PR and clicking the play icon.