Release 0.24 - March 2019 #6968

laurentlb · 2018-12-19T17:10:03Z

Target RC date: March 1st

katre · 2019-03-01T15:46:49Z

Looks like we have a good baseline for the release in 235e76b. Test run is here: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/831. @meteorcloudy has investigated the two failing projects and they are not release-related.

meteorcloudy · 2019-03-01T15:52:17Z

@oquenchil 235e76b won't contain your flip of --incompatible_disable_legacy_cc_provider

katre · 2019-03-01T15:57:26Z

The bazel-at-head-plus-downstream tests that include the flag flip for
incompatible_disable_legacy_cc_provider show many failures of downstream projects related to the missing "cc" provider (see https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/834). I think this feature needs more time to migrate downstream projects.

meteorcloudy · 2019-03-01T16:01:59Z

@katre Yes, @oquenchil and I just agreed to rollback the flag flip

katre · 2019-03-01T16:34:01Z

Creating rc1:

scripts/release/release.sh create 0.24.0 235e76b0e756d05599a6cbe1663ff8e13df84a86

Tests are running now.

katre · 2019-03-01T18:48:18Z

Release is blocked by #7605: I cannot actually get the release binaries to build.

meteorcloudy · 2019-03-04T10:55:52Z

@katre Thanks for adding migration-0.24 for the incompatible flags didn't make into 0.24.0.
But we should not remove migration-0.22 and migration-0.23 for them, because if we are using an old version of Bazel on CI we won't be able to fetch them for testing. (For example, we currently still uses 0.22.0 because 0.23.0 has a bug and 0.23.1 is not yet released)
https://buildkite.com/bazel/bazelisk-plus-incompatible-flags/builds/22#98b39372-548a-43be-827f-b096f70ab30e only got 7 incompatible flags, but there are more that should be tested.

I'm adding the migration labels back for 0.22 and 0.23

philwo · 2019-03-04T10:57:37Z

Thank you @meteorcloudy!

katre · 2019-03-04T13:23:46Z

@meteorcloudy My mistake. We should update the docs at https://github.com/bazelbuild/continuous-integration/blob/master/docs/release-playbook.md and clarify that (I think I see where I went wrong, so expect a PR from me today).

katre · 2019-03-04T13:41:00Z

Creating rc2:

scripts/release/release.sh create --force_rc=2 0.24.0 235e76b0e756d05599a6cbe1663ff8e13df84a86 badd82e4c5cda7b3232481e1c63a5550ac898cd8

vbatts · 2019-03-28T15:13:17Z

@hlopko fedora 30 is soon to release, and the current "rawhide" will be fedora 31 about 6 months later. I bet there will be a fix before then

werkt · 2019-03-28T17:17:20Z

I'd like to suggest that a 0.24.1 be released which incorporates changes from #7860 - it will not apply cleanly, as it comes after 9813c58, but it represents a serious bug in 24 tracked in #7856 that we'd much rather not wait for a cut of 25 for a fix of.

katre · 2019-03-28T17:23:12Z

@werkt Is that issue a release blocker? Asking @buchgr to help triage.

werkt · 2019-03-28T17:48:40Z

I'll defer to @buchgr on that, but we have been broken since 0.22 with this particular issue (0.23 introduced the break), are still broken on 0.24, won't be fixed until 0.25 without intervention, and have only had a chance to properly diagnose it now (we were late to adopt 0.23 as well).

katre · 2019-03-28T17:54:44Z

Makes sense for a patch release. I want to wait for the result on #7874 before I cut it, however.

hlopko · 2019-03-29T10:50:14Z

Oh wait gcc-9 is not yet released :) Ok I say #7874 doesn't warrant a patch release.

vbatts · 2019-03-29T12:05:23Z

On 29/03/19 03:50 -0700, Marcel Hlopko wrote: Oh wait gcc-9 is not yet released :) Ok I say [1]#7874 doesn't warrant a patch release.

yeah. Fedora Rawhide, is literally that. It's meant as an early canary. :-)

katre · 2019-03-29T15:47:03Z

SInce #7874 isn't critical, I will cut a patch release today that includes 80bff99 (to address #7856).

katre · 2019-03-29T16:26:27Z

I tried to create 0.24.1rc1:

./scripts/release/release.sh create --force_rc=1 0.24.1 f092ec388a4b58788285b928c86f0f81561d4be4 80bff99811356ea5ec2a66f0217df75a56923069

However, I cannot merge this cherry-pick: the code has changed too far in between and I have no idea what should actually be fixed.

@werkt and @buchgr Please give me directions (or, better yet, a commit I can apply to 0.24.0) on how to resolve this. Or wait for the 0.25.0 RC to be delivered early next week.

werkt · 2019-03-29T17:11:11Z

Working one up based on release-0.24.0 now

werkt · 2019-03-29T17:45:06Z

https://github.com/werkt/bazel/tree/retry-exception-fix-0.24 contains a single commit that addresses the issue and applies to the base commit of 0.24

katre · 2019-03-29T17:54:56Z

Okay, second attempt at 0.24.1rc1:

$ git branch -D release-0.24.1
$ git remote add werkt [email protected]:werkt/bazel.git
$ git fetch werkt
$ ./scripts/release/release.sh create --force_rc=1 0.24.1 f092ec388a4b58788285b928c86f0f81561d4be4 c56c489119e6587975964c44ceb9e429ad950736

Waiting on tests and binaries now.

katre · 2019-03-29T19:26:14Z

0.24.1rc1 is now out: https://releases.bazel.build/0.24.1/rc1/index.html
@werkt Please test this.

werkt · 2019-03-29T21:12:29Z

Tested, this correctly generates an error that would result in a fallback, excellent!

werkt · 2019-04-01T21:50:25Z

This release candidate looks acceptable from my end, is there any update on this?

katre · 2019-04-02T16:07:55Z

I am releasing it today (two business days have passed with no new release blockers).

katre · 2019-04-02T17:00:57Z

0.24.1 is now released (https://github.com/bazelbuild/bazel/releases/tag/0.24.1).

@petemounce, @vbatts, and @excitoon: can you update the packages you each maintain? Thanks!

vbatts · 2019-04-02T17:10:13Z

fedora and centos build: https://copr.fedorainfracloud.org/coprs/build/876626

petemounce · 2019-04-02T20:27:50Z

Pushed.

menny · 2019-04-03T00:11:50Z

Bazel crash immediately with error:

/var/lib/spotify/buildagent/teamcity/work/70b3e9bc32b57ee2/bin/bazel: relocation error: /var/lib/spotify/buildagent/teamcity/work/70b3e9bc32b57ee2/bin/bazel: symbol _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE, version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

On Linux gew1-buildagent-ssd-0933-d.gew1.spotify.net 3.16.0-77-generic #99~14.04.1-Ubuntu SMP Tue Jun 28 19:17:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

0.23.1 works fine.

katre · 2019-04-03T14:05:05Z

What version of glibcxx are you using?

@hlopko, is this something that is easy to fix?

hlopko · 2019-04-04T16:41:49Z

@menny, do you use your own C++ toolchain? Did you migrate for #6861, including enabling features that used to be enabled by default in the past:

dependency_file
random_seed
module_maps
module_map_home_cwd
header_module_compile
include_paths,
pic
preprocessor_define

hlopko · 2019-04-04T16:45:22Z

Oh ignore me, I misuderstood the issue.

katre · 2019-04-05T13:21:42Z

@menny @hlopko Please file this as a separate issue to ensure it isn't missed for 0.25.

hlopko · 2019-04-08T15:52:05Z

Hi @menny - is this still an issue? Can you repro with a clean install of Ubuntu 14.04 (docker will be fine)? Would you mind creating a new issue and pinging me there so we can investigate properly? Thanks!

menny · 2019-04-08T16:05:29Z

@hlopko, It's still an issue on some of our CI pools (the ones that have ubuntu 14).
I'll try to reproduce it on a Docker container.

katre · 2019-04-09T13:43:34Z

I am closing this issue since 0.24.1 is fully released. Once the followup issue about GLIBCXX_3.4.21 is triaged and we determine if a further patch to 0.24 is required, I will re-open.

menny · 2019-04-09T15:12:12Z

@hlopko I've decided to upgrade the old CI machine to Ubuntu 18.
The crash is not an issue any longer.

laurentlb added the release label Dec 19, 2018

laurentlb assigned philwo Dec 19, 2018

katre assigned katre and unassigned philwo Jan 17, 2019

laurentlb pinned this issue Feb 28, 2019

dslomov mentioned this issue Mar 1, 2019

incompatible_disable_objc_library_resources #7594

Closed

katre closed this as completed Apr 9, 2019

katre unpinned this issue Apr 9, 2019

Release 0.24 - March 2019 #6968

Release 0.24 - March 2019 #6968

Comments

laurentlb commented Dec 19, 2018

katre commented Mar 1, 2019

meteorcloudy commented Mar 1, 2019

katre commented Mar 1, 2019

meteorcloudy commented Mar 1, 2019

katre commented Mar 1, 2019

katre commented Mar 1, 2019

meteorcloudy commented Mar 4, 2019 • edited Loading

philwo commented Mar 4, 2019

katre commented Mar 4, 2019

katre commented Mar 4, 2019

vbatts commented Mar 28, 2019

werkt commented Mar 28, 2019

katre commented Mar 28, 2019

werkt commented Mar 28, 2019

katre commented Mar 28, 2019

hlopko commented Mar 29, 2019

vbatts commented Mar 29, 2019 via email

katre commented Mar 29, 2019

katre commented Mar 29, 2019

werkt commented Mar 29, 2019

werkt commented Mar 29, 2019

katre commented Mar 29, 2019

katre commented Mar 29, 2019

werkt commented Mar 29, 2019

werkt commented Apr 1, 2019

katre commented Apr 2, 2019

katre commented Apr 2, 2019

vbatts commented Apr 2, 2019 via email

petemounce commented Apr 2, 2019

menny commented Apr 3, 2019

katre commented Apr 3, 2019

hlopko commented Apr 4, 2019

hlopko commented Apr 4, 2019

katre commented Apr 5, 2019

hlopko commented Apr 8, 2019

menny commented Apr 8, 2019

katre commented Apr 9, 2019

menny commented Apr 9, 2019

meteorcloudy commented Mar 4, 2019 •

edited

Loading