CS2 Discussion: Question: Is conflating null and undefined a feature of CoffeeScript?

[coffeescriptbot]

From @GeoffreyBooth on 2016-10-19 07:15

In CoffeeScript 1.x, null and undefined are often interchangeable:

• if foo? returns false whether foo is null or whether foo is undefined.
• ((a = 1) -> a)(null) and ((a = 1) -> a)(undefined) both return 1.

In ES2015, at least regarding default function parameters, null and undefined are not interchangeable:

• (function (a = 1) { return a; })(null) returns null.
• (function (a = 1) { return a; })(undefined) returns 1.

This has sparked a discussion in the PR where I’m trying to implement arrow functions (and by extension, ES-style default parameters) for CoffeeScript 2. Basically, if we want to preserve the current behavior where the default value is applied when a parameter is null or undefined, we need to keep the current generated output:

f = function(a) {
  if (a == null) {
    a = 1;
  }
  return a;
}

So if we want output like this:

f = function(a = 1) {
  return a;
};

Then we have to break backward compatibility and go with ES’ implementation of default parameter values applying only when the parameter is undefined, not null or undefined. We can’t do both (the default value and the null if block) because the default value could be a function call, for example f = (a = b()) -> a.

CoffeeScript 2 was going to break backward compatibility anyway, but we want to break as little as possible. Is this null/undefined interchangeability something that people value in CoffeeScript 1.x, or would you not miss it if it went away? How important is it that the generated JavaScript for a feature like default function parameters be as ES-like as possible?

[coffeescriptbot]

From @lydell on 2016-10-19 07:55

In VimFx (~5k lines of CoffeeScript), undefined is never used as a literal – only null. If CoffeeScript were to break backwards compatibility regarding parameter defaults, I'd have to go through every function with parameter defaults (and all places the call those functions) and double-check all values that can possibly be null (to see if I need to change to undefined somewhere, which I most likely need to do in most cases). Being a crazy refactoring maniac, that wouldn't bother me personally very much though (other than a few regression bugs).

But for ? I think it really makes sense to check both null and undefined, because, to me, the point of using ? in the following code is to avoid exceptions: element?.getAttribute?('foo').

It would be nice to hear @jashkenas’ wisdom on this as well.

Is parameter defaults behaving different in CoffeeScript compared to ES a relief or a point of confusion for new CoffeeScript users? What about already experienced users?

[coffeescriptbot]

From @triskweline on 2016-10-19 08:01

While null and undefined are sometimes interchangeable, they often are not. E.g. == doesn't conflate these (and it should never do). It's OK to decide this on a per-case basis.

Conflate for existential foo? => I think that 's a very handy feature to have

Conflate for default arguments => I'd much rather have the cleaner Javascript output. Note that "The compiled output is readable" is also a CS feature, it says so right on the homepage!

In the PR discussion, I get @lydell saying that he doesn't care about the difference, got into the habbit of using null, and is now tripped up by ES semantics. I don't care either most of the time, but happen to default to undefined instead. That's a recommendation from Douglas Crockford in Javascript - The Better Parts.

[coffeescriptbot]

From @jashkenas on 2016-10-19 14:13

It's tricky. Having two slightly different values for nothing is absolutely a mistake in JavaScript, and it's something we tried to smooth over a bit in CoffeeScript my making undefined and null as interchangeable as possible.

But getting the clean output for the 2.X branch is also a desirable outcome.

I'd say that it's tragic, but go for the clean output for 2.0, and we'll live with the inconsistency.

[coffeescriptbot]

From @GeoffreyBooth on 2016-10-20 15:06

We should be keeping track of breaking changes somewhere. A wiki page on the main repo? A wiki page here? A markdown file in the project? Part of index.html in the project?

[coffeescriptbot]

From @lydell on 2016-10-20 15:19

Let’s start with a wiki page on the main repo.

[coffeescriptbot]

From @GeoffreyBooth on 2016-10-21 05:32

Wiki page here. I’ve updated #4311 to reflect this decision (assuming we’re going with ES syntax, at least for now).

[coffeescriptbot]

From @connec on 2016-10-21 17:37

Paraphrased from #4311 (wall of text incoming):

Full disclosure: I'm in the same bucket as lydell in that I've tended to use null rather than undefined, particularly as the last expression in procedures for returning an empty value. When I'm writing CS I like to live in a world where undefined does not exist.

I would rather lose ES2015's defaults and keep existence checks consistent. I really don't like the idea of defaults behaving differently to ?, ?=, etc. (though ?() gives precedent for that...). To illustrate a problem with this, currently you can use argument defaults to make values in functions 'safe' for existence checks, i.e. a default is as good as an existence check.

action = (options = {}) ->
  # We can safely call options.whatever here, since we know it's not undefined or null
  options.whatever

Admittedly this is also an issue with ES2015, but it's one that CoffeeScript solves by treating null and undefined as non-existent values, for existence checks and defaults. My issue isn't so much about allowing null as an empty argument, but more about just not having to think about the difference.

If clean output is such a desirable outcome, why not just skip CS and use ES2015? I appreciate that's a bit melodramatic, and CS adds much more value than just null/undefined unification, but it's one of the many CS features that make it so pleasingly consistent in contrast to JS' (especially ES2015's) syntax limitations and quirks.

Finally, I do feel we're being a bit flippant about changing CS semantics when there's no compatibility reason to do so, particularly when there's so much value in a release of CoffeeScript that fixes what's broken (classes) without breaking lots of backwards compatibility.

[coffeescriptbot]

From @jashkenas on 2016-10-21 17:43

Finally, I do feel we're being a bit flippant about changing CS semantics when there's no compatibility reason to do so, particularly when there's so much value in a release of CoffeeScript that fixes what's broken (classes) without breaking lots of backwards compatibility.

I agree — but I think that's basically the whole point of the current plan for "CoffeeScript 2". CoffeeScript 1 can continue to be the nice little well-tended garden that it has been ... but the point of CS2 is to embrace as much of modern JS as we can, even if that means losing some nice things that we've previously enjoyed.

[coffeescriptbot]

From @GeoffreyBooth on 2016-10-21 18:02

@connec does have a point about classes: some people might want to limit their breaking changes to “just” classes, to get the interoperability with ES classes that they need, and could care less about any other changes CS2 brings to the table. For such people it will be annoying to have to choose between 1.x’s lack of class interoperability and 2’s breaking changes that are unrelated to fixing classes.

As much as I hate to say it, we could release the classes fix as an off-by-default flag on 1.x, so that people have a way to opt-in to that breaking change without any of the others. The only other interoperability concern that 2 was planning to address was tagged template literals, but those can be implemented in 1.x too without breaking changes (whereas regular ES template literals, by contrast, need to be 2-only).

This might even be a way to thread the needle with regard to how we want to deal with classes in 2: the maximum-compatibility version, basically what @connec is working on in #4330, could be the opt-in ES-compatibility classes for 1.x; and a new implementation of classes that more closely tracks the ES spec could be what’s in 2. This sounds like a lot of work, and I’m not saying I recommend it, but it’s an option.

[coffeescriptbot]

From @jashkenas on 2016-10-21 18:14

Just for the record: I think that sort of muddled approach is the exact wrong way to go. It's already complicated, and the clarity of —

CoffeeScript 1 is the CoffeeScript you know.
CoffeeScript 2 is CoffeeScript brought as close to modern JavaScript as possible.

— is critical.

If you add a flag for classes in 1, next you're going to be adding a flag for async, and then for some sort of advanced module feature, and then another flag for ES destructuring ... and so on.

[coffeescriptbot]

From @GeoffreyBooth on 2016-10-21 18:54

I agree, but I wanted to throw it out there as an option. I suppose someone could always make a fork of CS1 plus #4330, if they really really wanted only the new classes but nothing else.

I guess the bigger question when it comes to “brought as close to modern JavaScript as possible” is, how close? We could disallow splats in function parameters that aren’t the final parameter, the way ES2015 does; that would certainly have made my PR far easier to implement. But that seemed like a pointless restriction to impose on CoffeeScript, even if that’s what ES requires.

I think we basically agreed in the other thread that prettier output alone is rarely good enough to justify a breaking change; but conforming to ES spec for parameter default values has a benefit beyond streamlined output, in that CoffeeScript would then behave more like ES2015 for people familiar with ES. It’s like if we add support for ...args in addition to args..., to make it easier for people coming from ES or going back and forth; there is some value in consistency with JavaScript, even if it means CoffeeScript itself is less internally consistent.

For the record, I think it’s worth conforming to ES on this issue at least. But I understand the complaint. Read up the thread of #4311 and you’ll see it took me some convincing as to which “old” features we were going to preserve, even if it meant outputting lots of extra JavaScript.

[coffeescriptbot]

From @connec on 2016-10-21 19:43

Apologies for another wall of text, but I'm an avid fan of CoffeeScript as it is today, and appreciate the decisions that were made in its design, particularly wrt. executable class bodies and conflating null and undefined, so here goes 😛

I feel like there's a balance of features that would satisfy people on both sides. There are plenty of features with compelling reasons to switch to ES2015:

• class is necessary to interact with ES2015 classes, which is critical for JS interoperability. If the final approach proposed in [wip] Compile classes to ES2015 #4330 is taken, the only breaking changes from this is the requirement of super calls in all derived constructors, and requiring constructors be called with new.
• Arrow functions correspond semantically to CS' fat arrow, so the handling of this can be passed to JS. This doesn't need to break anything apart from using this, though it does require different handling of parameters to deal with the absence of arguments.
• Destructuring in CS requires quite a few ref's and chained assignments, which can be cleaned up by compiling to JS destructuring. I'm a bit indifferent about this one, but it would be easy enough to adopt it without breaking any existing behaviour by continuing to do a bit of extra work around this and defaults, and it would make the common cases almost identical, syntactically.
• Dynamic object keys are a no-brainer (all gain, no pain?).
• Template literals are probably in the same category.

The main one here is definitely class, without that people literally can't use CS if they want interop with ES classes. The rest are nice-to-haves with the potential of simplifying the compiler and/or the compiled output. Regardless, I can't think of any major drawbacks for any of the above (correct me if I've missed any).

Once they are taken care of, there are other things where the main benefit is cleaner output at the cost of backwards compatibility within CS. Full class compilation, implementing splats, and default arguments would only be 'simple' if CS' semantics are changed (e.g. no more executable class bodies, no more rest before args, defaults not overriding null). These are certainly more controversial. My fundamental problem with adopting changes in this category is they take away features from the language without adding anything.

I do understand that reducing friction with ES will make CS a more compelling choice for people looking for something 'like JS, but prettier', but certainly in these cases (executable class bodies, rest args, and defaults) there would be very few surprises for users that couldn't be summed up in some short sentences:

• CoffeeScript, like Ruby, supports executable class bodies.
• Note that CoffeeScript allows splat arguments at any position in an arguments list.
• Note that CoffeeScript treats null and undefined as missing values for the purpose of default arguments, similar to the existential operators.

[coffeescriptbot]

From @GeoffreyBooth on 2016-10-22 18:31

I don’t feel strongly about this either way. The code is written for either implementation. Perhaps we need more opinions?

[coffeescriptbot]

From @leonexis on 2016-10-22 20:13

There is a lot to be considered here. I'm in the "ES but prettier" camp. I've fought and lost hours many times with CS1 mainly due to some of the fundamental differences between CS and ES. The only reasons why I stuck with it mainly included class support and easier-to-read code. I've looked at changing to ES2015+ because it now implements many of the features I wanted, but it is still ugly as sin. Readability is currently winning out for me, but nothing would make this developer happier than have some more cohesiveness between CS and ES. A lot of my lost time dealt with treating null and undefined the same in CS where I expected it to be treated differently coming from ES. For features that ES just didn't have (like the existential operator), I didn't care that it treated null and undefined the same, because that was not an ES feature, but something awesome and unique added in CS.

In short, if I'm writing something that looks like ES in CS, it should act like, and preferably output, ES. For example, if I use a rest argument like (foo, ...bar), it should act exactly the same in CS as ES. If I'm using a feature with no equivalent in ES (existential operator, or non-last-argument splat), then I don't mind how CS chooses to implement it. I understand that it's not an ES feature and am in CS-only territory.

In any case, all you guys are doing an amazing job. These are just my two cents.

[coffeescriptbot]

From @GeoffreyBooth on 2016-10-26 05:32

Update: I’ve merged in #4311 (arrow functions, including default parameters) following the ES spec, where a parameter default value is applied for undefined but not for null. This can easily be changed in the future if there’s a groundswell of support for the reverse. I think it might be best to err on the side of ES compatibility for now, at least until after getting some feedback from some larger projects that start trying to use CS2.

[coffeescriptbot]

From @leonexis on 2016-10-26 17:02

I like siding on ES compatibility for this stage of CS2, but I also agree with needing more input. The commercial projects I'm involved with only account for about a hundred thousand lines of code split between browser and server software, and I can't release any of that. Thats pretty small compared to the rest of the CS community.

It would be interesting to get feedback from the Atom.io developers since the Atom editor is mostly written in CoffeeScript. This is also a critical time since according to recent commit logs, Atom is converting some CoffeeScript to JavaScript. I've always used them as an example of a large scale project written in CS and would hate to loose them as a big supporter of CS.

[coffeescriptbot]

From @emptist on 2016-11-01 00:52

I believe those who seeking for similarity in CoffeeScript will finally turn to use pure JavaScript itself.

[coffeescriptbot]

From @danielbayley on 2016-11-26 15:19

Having two slightly different values for nothing is absolutely a mistake in JavaScript

Agreed!

It would be interesting to get feedback from the Atom.io developers since the Atom editor is mostly written in CoffeeScript. This is also a critical time since according to recent commit logs, Atom is converting some CoffeeScript to JavaScript. I've always used them as an example of a large scale project written in CS and would hate to loose them as a big supporter of CS.

Having written a bunch of @atom packages and being a big CS fan myself, I definitely feel the same way…

Penny for your thoughts @lee-dohm?

[coffeescriptbot]

From @JavascriptIsMagic on 2016-11-27 16:26

I have been writing my code in coffeescript at work for quite some time now, though I'm afraid most of it is closed source like leonexis.

There where two things that really drew me to coffeescript:
Syntax: Significant whitespace with the ability to eliminate the excessively abundant (()()) and function(){{}{}{}}.
And the ?. ?= ? existential operators. It's so intuitive when I don't want the possibility of an "of null" or "of undefined" TypeError. I just always intuitively assumed that default parameters in functions, and destructuring worked the same as the ?= operator.

I had some pressure at work to switch over to ES after the => was added, but the lack of the existential operator is actually what won me an argument to stay on coffeescript with my coworkers instead of migrating over to es6.

That being said, if we do need compatability with only undefined in the case of default perimeters, why not allow the ?= to be used in their place when we want the old behavior of conflating null?

((a ?= 1) -> a)(null) would return 1

((a = 1) -> a)(null) would return null and output formal ES default parameters

{a ?= 1} = {a:null} a would equal 1

{a = 1} = {a:null} a would equal null and output formal ES destructuring assignment

I am really sad to see that the ES spec got it wrong in my opinion, because one of the points of default parameters and destructuring assignment with defaults in my mind is to grantee that you won't be getting a TypeError if you try to use the . operator on those variables.

I know this does not solve the backwards compatibility with old code bases, but I would really miss the null and undefined conflation here!

[coffeescriptbot]

From @leonexis on 2016-11-27 16:44

I really love the option proposed by @JavascriptIsMagic. If it doesn't conflict with anything else internally, I think using ?= rather than = for the default CS1 behavior in destructuring assignments would be a great feature. It keeps ES behavior when written like ES and allows an easy to remember operator (?=) that is already well understood in CS for those that want to check for either null or undefined as it worked in CS1. Additionally, for someone reading CS2 who already knows a bit about CS1, it should be very apparent right away what (foo ?= 'bar') -> means with very little thought. It is a win-win from my perspective.

[coffeescriptbot]

From @GeoffreyBooth on 2016-11-27 21:11

?= in parameter default values was proposed here although it didn’t really get a full airing. @lydell or @vendethiel, do you care to weigh in on how you feel about this? In short:

• (a = 1) -> follows ES semantics: a is 1 only if a is undefined; and
• (a ?= 1) -> follows CS 1.x semantics: a is 1 if a is undefined or null

[coffeescriptbot]

From @vendethiel on 2016-11-27 22:05

I'm not sure it it should be ?= instead of ?, but that's nitpicking.

This one's a hard one. Even if we go that route, people will have to update all their = to ? anyway. And we don't want coffee to have them swapped (like we have in/of swapped).
The only horse I have in this race is the language itself. I do feel this one would be the most consistent and logical one, even tho it's quite the breaking change...

[coffeescriptbot]

From @vendethiel on 2016-11-27 22:06

(As I'm thinking about it, writing a tool that reports every instance to change should actually be doable)

[coffeescriptbot]

From @connec on 2016-11-27 23:11

Parroting myself a bit, but I still feel like the best choice would be to change default arguments to behave how they did before. They add a bit of verbosity to the compiled output when checking default arguments, but have a consistent definition of existence.

[coffeescriptbot]

From @JavascriptIsMagic on 2016-11-28 03:18

One thing I notice is that if we output to es6 default function parameters and destructuring that:

In CS1 (a = 1) -> a and -> [a = 1] = arguments and (a) -> a ?= 1 are basically the same.
In CS2 (a = 1) -> a and -> [a = 1] = arguments are (a) -> if a is undefined then a = 1 else a will be the same.
There is no operator equivalent for if a is undefined then a = 1 else a though you could use [a = 1] = [a] I suppose.

Are function defaults going to have the same behavior as destructuring default assignments?

One question I would have is should all assignment operators be valid? ?=, or=, +=, *=, /= etc...
I know I have wished I could use ((a or= "was falsy") -> a)(false) or {a or= "was falsy"} = {a:false} on more then one occasion though I don't know if this counts as fancy new sugar or not.

Just making ?= valid would at least allow for the old feature set to still be in the language.

---

If we only exclusively use ES6 syntax for default function parameters and for destructuring, I am very likely to stop using these features almost entirely because I will not be able to avoid risking a TypeError with null

[coffeescriptbot]

From @GeoffreyBooth on 2016-11-28 04:14

To keep things on track, for now at least we’ve made the decision that when it comes to CS2, we’re following ES semantics when it comes to features that ES has adopted from CoffeeScript. It’s too confusing for people coming from ES for function default parameters to behave one way in ES and a different way in CS. Ditto for destructuring when that is implemented. Not only is CoffeeScript “just JavaScript,” but one of the other core principles is that we generate concise, readable output; and compiling ES features one-to-one should be the expected behavior.

That doesn’t mean we can’t extend ES. In the CS2 version of fat arrow functions, we allow ... for any argument even though ES doesn’t; we allow this in parameters, e.g. (@a) => even though ES doesn’t; and so on. If people want the “null and undefined are conflated” behavior for function default parameters, I think the way to achieve that is by similarly extending ES, for example by allowing an operator such as ?=.

[coffeescriptbot]

From @jashkenas on 2016-11-28 17:41

I think for V2, in this case, we should just follow the ES undefined/null semantics. It's not ideal, but it's better than the additional complexity of introducing a new operator just for this niche case. You can work around it.

[coffeescriptbot]

From @vendethiel on 2016-11-28 18:28

I'm not sure I'd call it a new operator. ? is one we've had for years now. And coco/ls have long allowed (a ? b) -> in argslists.

[coffeescriptbot]

From @jashkenas on 2016-11-28 18:32

The problem is with object destructuring ... simple arguments would be alright, but introducing a separate form of equality into objects would get real ugly, real fast:

   {a: 1, b ?= 2} = c

Gross!

[coffeescriptbot]

From @GeoffreyBooth on 2016-11-30 23:00

I think the decision has been made for now: when it comes to function default parameters and destructuring, CoffeeScript 2 will conform to ES semantics and output idiomatic ES. If people want to propose sugar like ?= to achieve the “conflate null and undefined” semantics, please propose such things in new threads.

[coffeescriptbot]

From @carlsmith on 2017-01-28 05:29

Having two slightly different values for nothing is absolutely a mistake in JavaScript.

I disagree. In an ideal type system, there should be two Null type values. I would name them none and void, rather than null and undefined, as the type is named Null, and undefined is just long and negative.

There's a conceptual distinction between none, which is explicit and meaningful, and void, which is implicit and meaningless.

An API could specify that a value is either an instance of User or none if there is no User. In that case, none would be appropriate, because the value tells you something meaningful about state.

Some expressions, like x if false should evaluate to void, because the value itself is meaningless. The value only exists at all because expressions must evaluate to something. Similarly, in languages where functions implicitly return none, it should be void. Functions often meaningfully return none.

With function invocations, arguments that are void should try and fall back to a default, but arguments that are none should just be none.

To try and offer a concrete use case, when you use an interactive interpreter, the value of any expression is automatically printed, so you can just enter 1 + 1 and it'll print 2. You don't need to invoke print explicitly. When you enter an expression that evaluates to none, interpreters for languages that only have one Null type value will not print anything, to prevent useless, implicit return values being printed below any side effects. You would otherwise get stuff like this:

>>> print(1 + 1)
2
none
>>>

When you interactively use an API where none is an expected result, it's quite annoying that you have to infer none values from nothing happening. In languages where the distinction exists, interpreters can ignore expressions that evaluate to void, but still print none as a regular, meaningful value.

Admittedly, there are no super compelling use cases, but the conceptual distinction is still important enough to matter. I think Eich had the right idea about this. We obviously don't want NaN or 'undefined' slots in arrays that are not even equal to undefined, but having a simple Null type, with just none and void, seems ideal.

[coffeescriptbot]

From @jashkenas on 2017-01-30 16:15

Of course, I disagree — and the fact that you had to explain your intended philosophy of the distinction between none and void at such length is exactly the reason why.

Just because you've learned and internalized your preferred distinction between "intentional nothing" and "accidental nothing" doesn't mean that all of the other users, library writers and coworkers in your hypothetical language have done the same.

Besides, we already have false.

nothing is nothing is null is void is undefined is none.

[coffeescriptbot]

From @carlsmith on 2017-01-30 17:50

It's not really a hypothetical language. I was trying to generalise for all type systems.

JavaScript has null and undefined, so it's not a preferred distinction I learned and internalised. I was defending part of JavaScript's design, and proposing better names, not proposing a novel type system.

I didn't really get the point about having false.

Not being able to distinguish between a function explicitly returning null as a meaningful value, and just returning null implicitly, because it has to return something, can be a problem in practice. The interactive interpreter example was lengthy, but illustrated the problem.

If there's only one Null type value, and you had null if x, it would evaluate to null whether x is true or false. Another issue is that f(null) is the same as f() if the function takes args.

To be honest, I've had to take tons of morphine every day for months, so should probably not being debating language design with a language designer.

Thanks for your patience mate.

[coffeescriptbot]

From @rattrayalex on 2017-01-30 19:38

Wow, hope you're okay man!

[coffeescriptbot]

From @carlsmith on 2017-01-30 22:53

Cheers dude. I'm cool. My disc went in my spine. I'm ok if I take my meds, but they make it difficult to focus. It's annoying when I want to code, but it's not the end of the world.

I hope you're all well. Thanks for all the work you're doing. It's good to see 2.0.0 coming together.

Thanks again man.