-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CS2] Add webpack support #4501
Conversation
* Move Node.js-only code from src/coffee-script.coffee to src/index.coffee * Use lib/coffee-script/index.js as npm package's "main" script * Export CoffeeScript from src/browser.coffee * Set package.json's "browser" field to lib/coffee-script/browser.js (used by webpack as entry point) * Use lib/coffee-script/browser.js as bower package's "main" script
This is a lot of rejiggering for the webpack use case. CoffeeScript is primarily a Node.js compilation tool — not a browser tool. Our That said, a general reorganization of Node-specific parts to make webpack work sounds fine. Let's just keep the entry point for browser use as "browser". |
src/coffee-script.coffee
Outdated
exports.run = (code, options = {}) -> | ||
mainModule = require.main | ||
# Compile and execute a string of CoffeeScript | ||
exports.run = (code, options = {}) -> throw new Error 'Not implemented' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why export this if it just throws an error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea is that coffee-script.coffee
will contain only platform-independent code. Any platform-specific code should be moved elsewhere. Throwing an error in those "abstract" method prevents developers from accidentally require lib/coffee-script/coffee-script
and not knowing what's wrong.
This will be tricky in 2. Take a look at |
@jashkenas The npm package's main is I modified bower package's main to |
@GeoffreyBooth The Markdown-it library seems to be added via simple concatenation in branch 2's cakefile. The coffee-script part seems to be the same as master. We could basically apply the same diff to Line 146 of branch 2's Cakefile: - return require['./coffeescript'];
+ return require['./browser']; |
@jashkenas A little follow up on bower.json's main field. Bower Spec defines it as:
It suggests that build tools determine how to use "main" field. webpack uses it as entry point. See: https://webpack.github.io/docs/usage-with-bower.html So it should points to a browser-specific implementation for build tools to work properly. |
Kind of a good argument for us trying to leave out all dependencies, and just do the markdown code block detection ourselves. After all — we don't need to parse 98% of Markdown. The only thing we need to parse is code blocks. |
Just a thought: It would be trivial to setup webpack for building coffee-script browser bundle instead of And I happened to be using Markdown-it in my project too. It works well with webpack. |
As @jashkenas implies, we’re not looking to add dependencies, so adding Webpack to our build is almost surely a nonstarter. The reason Markdown-It was added was to address false positives in detecting Markdown non-code blocks: #3924. Though the original issue didn’t have examples that failed, so I don’t know what would be involved in rolling our own implementation. That’s perhaps something we should revisit. The docs do say:
So we should really try to honor this promise, and disentangle the Node dependencies. |
Webpack would be a devDependency, which is not the same as dependency. Anyway it's just a possibility and off the topic of this PR. So just ignore me. :) |
Markdown is very difficult to parse. |
@akfish A few questions:
I think coming up with a good test for this is important. Years ago the compiler didn’t depend on Node, and clearly at some point that got lost; if we have a test that fails the moment someone puts |
I'll cook up some example repos for 2 and 3. |
Check out https://github.com/akfish/coffeescript/tree/demo/webpack-bundle. This repo demonstrates the minimal modifications (webpack as devDependency and a new When running this test, I found out that generated |
Here is a minimal demo showing webpack using CoffeeScript: https://github.com/akfish/coffeescript-webpack-demo |
Cakefile
Outdated
if (typeof fs !== 'undefined' && fs !== null) | ||
source = fs""" | ||
# We don't need moduleMain, since the parser is unlikely to be run standalone | ||
parser = require('./lib/coffee-script/grammar').parser.generate(moduleMain: ->) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is brilliant. Thanks for solving this better than my hack.
Cakefile
Outdated
@@ -135,7 +129,7 @@ task 'build:browser', 'merge the built scripts into a single file for use in a b | |||
var CoffeeScript = function() { | |||
function require(path){ return require[path]; } | |||
#{code} | |||
return require['./coffee-script']; | |||
return require['./browser']; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you explain this? Why browser
here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is for logic consistency. Since browser.js
is already used as the entry for bower and webpack. It could avoid unnecessary confusion to use it for all browser related situations.
bower.json
Outdated
@@ -1,7 +1,7 @@ | |||
{ | |||
"name": "coffee-script", | |||
"main": [ | |||
"lib/coffee-script/coffee-script.js" | |||
"lib/coffee-script/browser.js" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is fine. I’ve used Bower, it’s only for frontend (browser-based) JavaScript.
src/browser.coffee
Outdated
@@ -4,7 +4,6 @@ | |||
# `text/coffeescript` script tags, source maps via data-URLs, and so on. | |||
|
|||
CoffeeScript = require './coffee-script' | |||
CoffeeScript.require = require |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for catching this.
src/browser.coffee
Outdated
@@ -18,6 +17,10 @@ CoffeeScript.run = (code, options = {}) -> | |||
options.shiftLine = on | |||
Function(compile code, options)() | |||
|
|||
CoffeeScript.register = -> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So it seems like the strategy here is to declare this dummy method, and then in the Node version this gets overridden with the real method? This is confusing, to say the least.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is confusing because register
method should not be available in browser environment, thus it should not be one of the abstract method. But doing so will break browser test.
Because coffee's browser test is not actually ran in a browser. It runs in node environment and the first call in the runTest
method is CoffeeScript.register()
. I guess this might be because there's no karma, PhantomJs nor headless chromium back then.
That's why I set it to a NOP implementation to pass the test for now. I could configure a test script that actually runs in browser with one of the methods mentioned above. But I think that should be a separate PR/issue for discussion since it requires adding more devDependencies.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On a separate note, I also think runScripts
method in the browser.coffee
could be used as browser's register
implementation. It's logically sound. But it will only work if the browser tests are actually running in browser. Right now it will only break the test when trying to call it in node environment.
What if we detected whether we’re in Node or not, and had the code branch accordingly? Or to follow the feature detection pattern, the functions that require Node modules could explicitly check for them and error out otherwise? Something like: try
fs = require 'fs'
path = require 'path'
module = require 'module'
vm = require 'vm'
catch (e)
exports.run = (code, options = {}) ->
unless require?.main? and process?.argv? and fs? and path? and module?
throw new Error 'This function requires CommonJS and the process, fs, path and module modules'
# rest of old `run` method And then we don’t need abstract or dummy methods, and the refactoring doesn’t need to be so dramatic. That would make me much more comfortable that we aren’t breaking downstream projects that might require the |
@GeoffreyBooth Tools like webpack use static analysis to find With that said, abstract method is one of the common patterns to enforce or at least document shared interfaces/behaviors, should multiple implementations are required. |
Hmf. So it tries to resolve all # `isNode` defined above somehow
exports.run = if isNode then require('run') else -> |
@GeoffreyBooth Yes. IIRC, webpack performs static analysis on AST. Older tools like browserify |
What’s the argument for merging this into I can see how having a browser-safe module you can include that generates ES3 is desirable; is that the appeal? I assume it wouldn’t be too difficult to configure Webpack to require CoffeeScript 2 (as modified by this PR) and pipe its output through Babel, even in a browser-compatible way? |
I have no problem with that, as long as it'll will be published to npm. I haven't had the need to generate ES3 yet so I can't say for sure. But it should be the case. |
CoffeeScript 2 is already on NPM: So if that's okay with you, please target the |
That’s a good idea. We actually have tests to run in an real browser environment: test.html. These tests don’t run as part of any The current test suite runs in about 2 seconds on my machine, and the Webpack test runs in about 1 second. Obviously this Docker test would take considerably longer, so it might make sense to have separate testing tasks like:
I’m not asking you to write the Docker test. But let’s put the Webpack test inside a task named As to your point about the Webpack output or build report: in this project, passing tests are silent and failing tests dump copious debugging info. I think the Webpack test should follow that pattern. You can add a comment explaining how to get the Webpack output in all cases (presumably just by running the Webpack command, I assume). That should be enough. So Webpack is generating this |
Since webpack support is actually support for a new build target, rather than a new feature, how about keep it consistent with other two existing build targets (node and browser), by add two new tasks:
In this way we can verify this new build target in two stages: 1) it should build and 2) built bundle should pass tests. Should the test configuration prove to be too much work to be complete in a single PR, we could at least have |
As for the browser test, docker is too heavy for this in my opinion. A popular choice would be karma. I used it for my past projects. It only requires developers to have browser installed (and who doesn't). The test output is piped to console with proper exit code so it's great for automatic tests. You can run it in Travis CI too. Though we will need to write some adapter code (or migrate tests to an existing test framework) for it to work, since this project doesn't use an existing one. The benefit is that we can run tests in actual browser environment, while sharing the same test suites among 3 build targets. Apparently browser test setup is another can of worms to be opened and deserves a separate PR. In fact we already have test coverage for webpack build from |
I renamed |
What purpose does |
Logical consistency for one. Both node and browser build target follows the 2 stage build/test process, why not for webpack? It is a build target and should be verified in both stages. |
I’m trying to avoid creating more It’s probably best to just have |
Testing a build target is not the same thing as testing a feature. Build report is not the same as test report and should not be consumed the same way. Discarding the build result doesn't make it less of a build target. And the webpack build result is not discarded: the build report is a build artifact and it is used. It is common for projects to have build tasks but not to distribute the build artifacts. Many Linux packages only provide source code and let the user A developer who's familiar with webpack would expect it to be treated as a build target. It could be more confusing to see it not to be treated as such, than having two tasks that keeps the logical consistency and can be easily documented. The logic is very clear: each build target would have Speaking of which, maybe I should add a |
A “webpack build” is not a feature of CoffeeScript, and I’m not interested in adding it as one. Just because a test involves running a build command doesn’t mean it’s not still a test. |
It's not a feature, it's a build target. Anyone familiar with webpack will not even come close to make the mistake you are thinking of. Anyway, enough with the terminology stuffs, as long as the webpack build command is not run as a unit test and the build report is not silent, I'm happy with it. If that's acceptable for you, I will make the modifications so that there's only one cake task |
Apologies. Looks like I should have helped put this ticket out of its misery long ago... CoffeeScript already works just fine with Webpack, and should not be restructured like this. I just walked through webpack's tutorial, using the CoffeeScript "browser" compiler, and it works perfectly. Using the latest version of CoffeeScript, you build the browser compiler:
Copy it into your webpack project, and then use it: import CoffeeScript from './coffee-script.js';
function component () {
var element = document.createElement('div');
element.innerHTML = CoffeeScript.compile('a = 1 + 1');
return element;
}
document.body.appendChild(component()); Index.html: <html>
<head>
<title>webpack 2 demo</title>
</head>
<body>
<script src="dist/bundle.js"></script>
</body>
</html> Then you run webpack:
And it works: @GeoffreyBooth -- if there's a change we need to make here, it's that we should be shipping the "browser" build file in a reasonable location as part of the npm install — you shouldn't have to build it yourself. Perhaps in a |
It works is not the same as it works "fine". I am aware of the setup you provided above. It shows how webpack supports a legacy project like this, not the other way around. The fact that the webpack support issue has been raised over and over by multiple people on multiple occasions apparently means nothing. Because people who use webpack on daily basis are neither too lazy to read the basic document, nor too stupid to know the difference. We should know better. Thanks for your time. |
....unbelievable |
Yes I think what @akfish is getting at is that you can’t just do Currently the browser build is treated as just part of the documentation, but clearly that needs to change. I think it deserves enough prominence that it is included in the NPM module, and What @akfish was accomplishing with this PR, which we might want to still consider, is that this PR enables people to do Maybe that’s not enough of an upside to be worth the changes contemplated by this PR, but as far as I can tell I don’t see any downside to these changes. It’s just a lot of lines of code moving around, but no functionality or clarity lost as a result. I guess if it provides a bit of convenience for Webpack/Browserify users, especially when it comes to time saved as they can avoid needing to troubleshoot errors caused by |
Looking through the diff a little more, that sounds fair enough — as long as we aren't contorting the file structure in any webpack-specific way. Go for it. I'm sorry if I offended. But we should still be offering the basic "coffeescript/browser" file as a general part of our packaging — so that folks who want to load it under a script loader of any stripe, or just vanilla JS, can get at it. |
Clientside CoffeeScript is useful in some circumstances, even if it's normally a mistake to use it that way. If you're using CoffeeScript in the browser for the right reasons, then you may not even have Node set up in your development environment. This PR is more to do with how the compiler is modularised for browsers, but having first-class support for clientside CoffeeScript is worthwhile. |
Agreed. @akfish, is my explanation of this PR’s value correct? Leaving aside the discussion of tests. |
@jashkenas can you please reopen? |
Looks like the repo/branch has been removed so the PR can't be reopened. |
Sorry for that. I should not have deleted the branch before the decision is final. I understand that the compiler usually don't mingle with webpack in this specific way thus there are some difficulties in communication. @GeoffreyBooth has put a lot of efforts into this and from his last comments I can see that we are on the same page. |
Per discussions with @GeoffreyBooth, @timhuff and @connec in #4276 #4277 reported by @Radivarig, current coffee-script source cannot be packed with webpack due to references to Node.js-only modules in main script.
Example error output of webpack:
Changes
src/coffee-script.coffee
tosrc/index.coffee
lib/coffee-script/index.js
as npm package's"main"
scriptCoffeeScript
fromsrc/browser.coffee
package.json
's"browser"
field tolib/coffee-script/browser.js
(used by webpack as entry point)lib/coffee-script/browser.js
as bower package's"main"
scriptTests
cake build
cake build:browser
cake test
cake test:browser