mk: Switch rustbuild to the default build system #37817

alexcrichton · 2016-11-16T23:20:45Z

This commit switches the default build system for Rust from the makefiles to
rustbuild. The rustbuild build system has been in development for almost a year
now and has become quite mature over time. This commit is an implementation of
the proposal on internals which slates deletion of the makefiles on
2017-02-02.

This commit also updates various documentation in README.md,
CONTRIBUTING.md, src/bootstrap/README.md, and throughout the source code of
rustbuild itself.

rust-highfive · 2016-11-16T23:20:48Z

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2016-11-16T23:21:41Z

r? @brson

Note that this shouldn't be r+'d yet as I'd like to wait for appveyor/Travis to go green (trying to run all the builders there).

Also if there's any more requests for documentation anywhere, I'd be more than happy to oblige!

petrochenkov · 2016-11-16T23:42:04Z

CONTRIBUTING.md

+To learn more about the driver and top-level targets, you can execute:
+
+```sh
+python x.py --help


I run x.py --help and it starts downloading something big from the internet instead of showing help.
I don't think this is an appropriate behavior.

Tracked here: #37305

petrochenkov

Fix #37305 before merging

petrochenkov · 2016-11-17T00:02:39Z

CONTRIBUTING.md

+of the compiler. You can execute it as:
+
+```sh
+python x.py build


Is bootstrap python3-compatible, btw? In case python defaults to 3. This is probably worth mentioning.
(Or build fails in some other way if python == python3, but python2 is still available? I don't remember exactly.)

Yes all the Python code in Rust repo is Py3k-compatible AFAIK, which is not so much btw. The whole Python 2-only thing in README is outdated actually.

Last I heard, Python 2 was only necessary for LLVM.

@retep998 Yeah you're right, there is this which means precisely Python 2.7.x is required, but rustbuild itself isn't limited to that. Which means compilation will only fail on systems with no Python 2.x at all due to LLVM, but should be fine otherwise.

AFAIK bootstrap.py is fully compatible, and then later version detection happens in rustbuild itself.

petrochenkov · 2016-11-17T00:12:45Z

configure

+    warn "the makefile-based build system is deprecated in favor of rustbuild"
+    msg ""
+    msg "It is recommended you avoid passing --disable-rustbuild to get your"
+    msg "build working as the makefiles will be deleted on 2016-01-02. If you"


Sorry for a silly question, but is the date in standard format (2 January) or in Freedom Units (1 February)?.

I just commented on the internals thread -- I think it should actually be February 2nd.

In any case, the year is wrong. (In the PR description, too)

petrochenkov · 2016-11-17T00:20:23Z

src/bootstrap/lib.rs

+//! to execute each in sequence. Each time rustbuild is invoked, it will simply
+//! iterate through this list of steps and execute each serially in turn.
+//! Rustbuild relies in each step internally being incremental and also
+//! internally parallelizing.


Could you document explicit parallelizing with -jN somewhere as well?

cuviper · 2016-11-17T00:36:23Z

configure

+    warn "the makefile-based build system is deprecated in favor of rustbuild"
+    msg ""
+    msg "It is recommended you avoid passing --disable-rustbuild to get your"
+    msg "build working as the makefiles will be deleted on 2016-02-02. If you"


retep998 · 2016-11-17T05:33:05Z

CONTRIBUTING.md

+python x.py build
+```
+
+If you're on a Unix-based system, you can also use the shorter version:


Technically you can also use a different short version on Windows if you have .py files associated with Python. x.py build in cmd and .\x.py build in powershell.

And MSYS2 is just "Unix-based" enough to see the shebang :D

retep998 · 2016-11-17T05:34:16Z

README.md


 ```sh
-$ ./configure
-$ make && make install
+$ python x.py build


This prompt is supposed to be cmd.exe according to the above paragraph, so using $ is incorrect here.

pitdicker · 2016-11-17T07:08:05Z

CONTRIBUTING.md

-questions on the build system internals, try asking in
-[`#rust-internals`][pound-rust-internals].
+The build system lives in [the `src/bootstrap` directory][bootstrap] in the
+project root. Our build system its itself written in Rust and is based on Cargo


pitdicker · 2016-11-17T07:13:32Z

src/bootstrap/lib.rs

+//!
+//! ## Architecture
+//!
+//! Although this build system defers most of the compilcated logic to Cargo


*complicated

tho it is also pretty compile-cated

pitdicker · 2016-11-17T07:20:13Z

src/bootstrap/step.rs

+    // That logic is all defined in the `native` module so we just delegate to
+    // the relevant function there. The argument to the closure passed to `run`
+    // is a `Step` (defined below) which encapsulates information like the
+    // stage, host, target, etc.


*compiled (I guess) line 77

pitdicker · 2016-11-17T07:31:34Z

src/bootstrap/lib.rs

+//!
+//! Although this build system defers most of the compilcated logic to Cargo
+//! itself, it still needs to maintain a list of targets and dependencies which
+//! it can itself perform. Rustbuild is comprised of of a list of rules with


*perform on. Rustbuild is made up of

Just a suggestion 'of of' reads a bit difficult

pitdicker · 2016-11-17T07:36:31Z

src/bootstrap/lib.rs

+//! to execute each in sequence. Each time rustbuild is invoked, it will simply
+//! iterate through this list of steps and execute each serially in turn.
+//! Rustbuild relies in each step internally being incremental and also
+//! internally parallelizing.


*For each step Rustbuild relies on the step to be internally incremental and also to be internally parallelizing.

alexcrichton · 2016-11-17T15:27:10Z

@petrochenkov can you elaborate on why you think #37305 is a blocker for landing this PR? I unfortunately don't see any path forward to solving it, so any opinions on how to solve it would also be greatly appreciated!

alexcrichton · 2016-12-05T04:39:28Z

@bors: r=brson

bors · 2016-12-05T04:39:29Z

📌 Commit 4755f95 has been approved by brson

bors · 2016-12-05T06:45:53Z

⌛ Testing commit 4755f95 with merge 0ae2037...

bors · 2016-12-05T06:48:50Z

💔 Test failed - auto-linux-64-x-android-t

bors · 2016-12-05T14:25:00Z

☔ The latest upstream changes (presumably #38100) made this pull request unmergeable. Please resolve the merge conflicts.

This commit switches the default build system for Rust from the makefiles to rustbuild. The rustbuild build system has been in development for almost a year now and has become quite mature over time. This commit is an implementation of the proposal on [internals] which slates deletion of the makefiles on 2016-01-02. [internals]: https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368 This commit also updates various documentation in `README.md`, `CONTRIBUTING.md`, `src/bootstrap/README.md`, and throughout the source code of rustbuild itself. Closes rust-lang#37858

alexcrichton · 2016-12-07T08:30:30Z

@bors: r=brson

bors · 2016-12-07T08:30:31Z

📌 Commit 0e272de has been approved by brson

bors · 2016-12-07T12:04:12Z

⌛ Testing commit 0e272de with merge 7846610...

mk: Switch rustbuild to the default build system This commit switches the default build system for Rust from the makefiles to rustbuild. The rustbuild build system has been in development for almost a year now and has become quite mature over time. This commit is an implementation of the proposal on [internals] which slates deletion of the makefiles on 2017-02-02. [internals]: https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368 This commit also updates various documentation in `README.md`, `CONTRIBUTING.md`, `src/bootstrap/README.md`, and throughout the source code of rustbuild itself.

bors · 2016-12-07T16:27:42Z

frewsxcv · 2016-12-08T21:02:18Z

It looks like Travis started failing everything after this pull request merged.

…rson Delete the makefile build system This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild. Rustbuild has been the default build system [since 1.15.0](rust-lang#37817) and the makefiles were [proposed for deletion](https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368) at this time back in November of last year. And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!

Delete the makefile build system This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild. Rustbuild has been the default build system [since 1.15.0](#37817) and the makefiles were [proposed for deletion](https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368) at this time back in November of last year. And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!

…rson Delete the makefile build system This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild. Rustbuild has been the default build system [since 1.15.0](rust-lang#37817) and the makefiles were [proposed for deletion](https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368) at this time back in November of last year. And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!

Delete the makefile build system This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild. Rustbuild has been the default build system [since 1.15.0](rust-lang/rust#37817) and the makefiles were [proposed for deletion](https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368) at this time back in November of last year. And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!

rust-highfive assigned aturon Nov 16, 2016

rust-highfive assigned brson and unassigned aturon Nov 16, 2016

petrochenkov reviewed Nov 16, 2016

View reviewed changes

petrochenkov requested changes Nov 16, 2016

View reviewed changes

petrochenkov reviewed Nov 17, 2016

View reviewed changes

alexcrichton force-pushed the rustbuild-default branch from bd5bab4 to 6b11d6a Compare November 17, 2016 00:33

cuviper reviewed Nov 17, 2016

View reviewed changes

retep998 reviewed Nov 17, 2016

View reviewed changes

pitdicker reviewed Nov 17, 2016

View reviewed changes

alexcrichton force-pushed the rustbuild-default branch 2 times, most recently from 8204cb4 to b92e513 Compare November 17, 2016 15:22

alexcrichton mentioned this pull request Nov 17, 2016

Buildbot grammar #37825

Closed

petrochenkov mentioned this pull request Nov 17, 2016

bootstrap.py --help starts with downloading #37305

Closed

alexcrichton force-pushed the rustbuild-default branch from b92e513 to b781dec Compare November 17, 2016 19:22

alexcrichton mentioned this pull request Nov 18, 2016

Rustbuild on 32-bit MSVC fails run-make tests on AppVeyor #37858

Closed

alexcrichton force-pushed the rustbuild-default branch 4 times, most recently from d5bfc7c to 3a66312 Compare November 18, 2016 19:57

alexcrichton force-pushed the rustbuild-default branch from 76f9e9e to 4755f95 Compare December 5, 2016 04:39

alexcrichton force-pushed the rustbuild-default branch from 4755f95 to fd183a2 Compare December 7, 2016 08:29

alexcrichton force-pushed the rustbuild-default branch from fd183a2 to 0e272de Compare December 7, 2016 08:30

bors merged commit 0e272de into rust-lang:master Dec 7, 2016

bors mentioned this pull request Dec 7, 2016

add preliminary support for incremental compilation to rustbuild.py #38072

Merged

alexcrichton deleted the rustbuild-default branch December 19, 2016 20:45

alexcrichton mentioned this pull request Jan 31, 2017

Delete the makefile build system #39431

Merged

mk: Switch rustbuild to the default build system #37817

mk: Switch rustbuild to the default build system #37817

Conversation

alexcrichton commented Nov 16, 2016 • edited Loading

rust-highfive commented Nov 16, 2016

alexcrichton commented Nov 16, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

petrochenkov left a comment

Choose a reason for hiding this comment

petrochenkov Nov 17, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pitdicker Nov 17, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcrichton commented Nov 17, 2016

alexcrichton commented Dec 5, 2016

bors commented Dec 5, 2016

bors commented Dec 5, 2016

bors commented Dec 5, 2016

bors commented Dec 5, 2016

alexcrichton commented Dec 7, 2016

bors commented Dec 7, 2016

bors commented Dec 7, 2016

bors commented Dec 7, 2016

frewsxcv commented Dec 8, 2016

alexcrichton commented Nov 16, 2016 •

edited

Loading

petrochenkov Nov 17, 2016 •

edited

Loading

pitdicker Nov 17, 2016 •

edited

Loading