RFC: introduce the flavor syntactic design pattern

[nikomatsakis]

Summary

This RFC unblock the stabilization of async closures by committing to K $Trait (where K is some keyword like async or const) as a pattern that we will use going forward to define a "K-variant of Trait". This commitment is made as part of committing to a larger syntactic design pattern called the flavor pattern. The flavor pattern is "advisory". It details all the parts that a "flavor-like keyword" should have and suggests specific syntax that should be used, but it is not itself a language feature.

In the flavor pattern, each flavor is tied to a specific keyword K. Flavors share the "infectious property": code with flavor K interacts naturally with other code with flavor K but only interacts in limited ways with code without the flavor K. Every flavor keyword K should support at least the following:

• K-functions using the syntax K fn $name() -> $ty;
• K-blocks using the syntax K { $expr } (and potentially K move { $expr });
• K-traits using the syntax K $Trait;
  • K-flavored traits should offer at least the same methods, associated types, and other trait items as the unflavored trait (there may be additional items specific to the K-flavor). Some of the items will be K-flavored, but not necessarily all of them.
• K-closures using the syntax K [move] |$args| $expr to define a K-closure;
  • Closures implement the K Fn traits.

Some flavors rewrite code so that it executes differently (e.g., async). These are called rewrite flavors. Each such flavor should have the following:

• A syntax 🏠K<$ty> defining the K-type, the type that results from a K-block, K-function, or K-closure whose body has type $ty.
  • The 🏠K<$ty> is a placeholder. We expect a future RFC to define the actual syntax. (The 🏠 emoji is meant to symbolize a "bikeshed".)c
• A "do" operation that, when executed in a K-block, consumes a 🏠K<$ty> and produces a $ty value.
• The property that a K-function can be transformed to a regular function with a K-flavored return type and body.
  • i.e., the following are roughly equivalent (the precise translation can vary so as to e.g. preserve drop order):
    • K fn $name($args) -> $ty { $expr }
    • fn $name($args) -> 🏠K<$ty> { K { $expr } }

Binding recommendations

Existing flavor-like keywords in the language do not have all of these parts. The RFC therefore includes a limited set of binding recommendations that brings them closer to conformance:

• Commit to K $Trait as the syntax for applying flavors to traits, with the async Fn, async FnMut, and async FnOnce traits being the only current usable example.
• Commit to adding a TBD syntax 🏠async<$ty> that will meet the equivalences described in this RFC.

Not part of this RFC

The Future Possibilities discusses other changes we could make to make existing and planned flavors fit the pattern better. Examples of things that this RFC does NOT specify (but which early readers thought it might):

• Any form of "effect" or "flavor" generics:
  • Flavors in this RFC are a pattern for Rust designers to keep in mind as we explore possible language features, not a first-class language feature; this RFC also does not close the door on making them a first-class feature in the future.
• Whether or how async can be used with traits beyond the Fn traits:
  • For example, the RFC specifies that if we add an async flavor of the Read trait, it will be referred to as async Read, but the RFC does not specify whether to add such a trait nor how such a trait would be defined or what its contents would be.
• How const Trait ought to work (under active exploration):
  • The RFC only specifies that the syntax for naming a const-flavored trait should be const Trait; it does not specify what a const-flavored trait would mean or when that syntax can be used.
• What specific syntax we should use for 🏠K<$ty>:
  • We are committed to adding this syntax at least for async, but the precise syntax still needs to be pinned down. RFC #3628 contains one possibility.

Rendered

[clarfonthey]

So, I understand why you're referring to these as colours, since "coloured functions" is what the larger programming language community uses as a term, but I think that the naming of effects should be used instead, especially since that's at least what the current Rust WGs have been settling on.

Can you think of an example of a "colour" by this metric that wouldn't be an effect? And even then, I think it would be say that the colour should have an effect on everything it labels, so, that meaning would still apply.

The biggest benefit to this is that "effects" also clearly evoke the purpose of the colouring: the async effect means that everything is designed for an async context, and the meaning is that return values are wrapped in futures. Similarly, the const effect means that everything is designed to be evaluable in const context.

Also just as a small picky addition: that bicycle emoji is impossibly small without zooming in and while I appreciate the joke, it is unnecessarily opaque and adds in extra effort where the word "bike" or "bicycle" or "bikeshed" would probably be a lot clearer.

[nikomatsakis]

I avoided the term "effect" for a few reasons.

One of them is that I think that it is overall kind of jargon. What's more, my observation is that it is divisive jargon, as people bring pre-conceived notions of what it ought to mean, and not everything that I consider a "color" fits into those notions.

My take on an effect is that it is some kind of "operation" that occurs during execution, such as a write to a specific memory region, a panic, a memory allocation, etc. It's reasonable to model this kind of effect as a "function" you can call when that event occurs (perhaps with some arguments).

From what I can tell, this definition lines up with Koka (which is a very cool language). However, Koka is also (I believe) based on Continuation Passing Style, which means that simple function calls get a lot more power. This allows them to model e.g. generators or exceptions as effects.

To my mind, this is kind of cheating, or at least misleading. In particular, we can't "just" port over Koka's abstractions to Rust because we also have to account for rewrites.

In any case, I'm open to a terminology discussion, though personally I'd be inclined not to rename colors to effects, but perhaps to rename filter colors to effect colors or effect-carrying colors.

EDIT: (added as a FAQ)

[rpjohnst]

Koka's use of CPS is not at all fundamental to its semantics of effects- it is an implementation detail. The semantically-interesting thing, shared in common with other languages' "(algebraic) effects," even when they are not implemented via CPS, is that those effect operations can be handled by suspending and resuming the program.

Rust also does the same thing via a different implementation strategy, a related lowering to state machines. It would be perfectly legitimate to call things like .await or yield or yeet "effect operations" regardless of their implementation strategy.

[programmerjake]

one more effect/color came up: strictfp (or whatever you want to call it) -- it is an opt-in allowing code to run with non-default fp environment.

[clarfonthey]

I avoided the term "effect" for a few reasons.

One of them is that I think that it is overall kind of jargon. What's more, my observation is that it is divisive jargon, as people bring pre-conceived notions of what it ought to mean, and not everything that I consider a "color" fits into those notions.

My take on an effect is that it is some kind of "operation" that occurs during execution, such as a write to a specific memory region, a panic, a memory allocation, etc. It's reasonable to model this kind of effect as a "function" you can call when that event occurs (perhaps with some arguments).

I mean, that is fair that it does give you some sort of preconceived notion, although I think that we've done a pretty good job reconfiguring some of these terms in Rust. To me, both "colour" and "effect" are equally jargon, but "effect" gives me at least an idea of what the jargon means, whereas "colour" gives me nothing at all. Like, if you're going to go with jargon at all, then you should at least try to use jargon that gives people a basic idea that they can latch onto after they know what the jargon means— this is why we use terms like "result" instead of "flargleflorpus" in Rust.

I would also additionally like to challenge that "effect" strongly confers the notion that something occurs at runtime, although we could make everyone upset and use "affect" instead. To me, this merely affects the things that are labelled with it, which… actually, that's two arguments for "affect" over "effect"…

[Lokathor]

I can definitely say that "color" is absolutely a jargon word when used for functions.

Separately, the bike emoji is cute to put in the RFC, but it doesn't render very well all the time. On my particular device, it's a small gray bike in a gray background tele-type text span, and it ends up almost disappearing from view unless you know you're looking for it. Might want to use an actual word like "bikeshed" instead.

[programmerjake]

Separately, the bike emoji is cute to put in the RFC, but it doesn't render very well all the time. On my particular device, it's a small gray bike in a gray background tele-type text span, and it ends up almost disappearing from view unless you know you're looking for it.

maybe use a different bicycle character: 🚴 -- it likely renders with more color.
(i tried using VS15 (U+FE0E) for the text style variant, but that seems to no longer work on my android phone, iirc i saw something about unicode deprecating that -- turns out it is a bug that was just recently fixed in chromium https://issues.chromium.org/issues/40628044)

[nikomatsakis]

@rpjohnst

The semantically-interesting thing, shared in common with other languages' "(algebraic) effects," even when they are not implemented via CPS, is that those effect operations can be handled by suspending and resuming the program.

This is what I was referring to, yes. Basically, doing those transformations in an omnipresent way. This has more to do with what it would mean for us to support "user-defined" colors than anything else (which obviously is way out of scope, especially for this RFC.)

[Diggsey]

This is... a lot to take in.

This RFC proposes a lot of magic but doesn't provide a concrete example (eg. with async). It describes being able to apply "async" to get a variation of a trait, but I'm not convinced that's... good? For example AsyncRead is already defined by both tokio and futures crates, and neither of them is a mechanical translation of the non-async Read trait. async-std was an experiment in a more direct translation of the standard library to the async world, and I think it succeeded in showing that wasn't good enough - tokio has been so successful partly because it tailors things for the async use-case.

Also, these colours are described as "rewriting" the item in question, but a big problem with Rust's async is that we can't even express many of the types (like async closures) yet, so what exactly are these types going to be rewritten into?

[kennytm]

Will this RFC or the future consider commitment about multi-colored items

1. multi-colored blocks does not seem worthwhile. K9 { K7 { K8 { $expr } } } is clearer than K4 K6 K5 { $expr } in emphasizing the application order.
2. multi-colored functions K1 K2 K3 fn $name() needs to be supported, given the precedence of const unsafe fn and async unsafe fn, and unlike the blocks you can't really break it into smaller parts
   • currently the order of the existing colors are fixed: const async unsafe fn $name(). But with more rewriting color the order does make a difference (try { async { e } } ≠ async { try { e } }). Meaning that async try fn f() and try async fn f() should both be allowed and result in different types when calling f(). And the current strict const async unsafe order should be relaxed.
   • needs to decide whether calling K1 K2 K3 fn f() -> T produces 🚲K1<🚲K2<🚲K3<T>>> or 🚲K3<🚲K2<🚲K1<T>>>
   • does async async async async fn f() make sense?
   • or one could require that an item can only take at most 1 rewrite color, but it seems unnecessarily restrictive.
   • can filtering color collapse i.e. const const const const unsafe unsafe unsafe unsafe fn f() be acceptable and considered equivalent to const unsafe fn f()
3. multi-colored closure K1 K2 K3 move || {} follows the same principle of multi-colored functions
4. multi-colored traits K1 K2 K3 FnOnce() 🤔

[rpjohnst]

But with more rewriting color the order does make a difference (try { async { e } } ≠ async { try { e } }). Meaning that async try fn f() and try async fn f() should both be allowed and result in different types when calling f().

This is totally avoidable and probably should be avoided. This has already been the subject of a lot of discussion: for example, see boats' https://without.boats/blog/poll-next/ and my https://www.abubalay.com/blog/2024/01/14/rust-effect-lowering

The general idea is that effect-carrying colors do not really make sense to compose via nesting. This is less obvious with try + async because you can get the behavior you want from impl Future<Output = Result>, but Result<impl Future> is already fairly useless, and it really breaks down when you compose multiple colors that can suspend and resume. For example neither impl Iterator<Item = impl Future> nor impl Future<Output = impl Iterator> are really workable, which is why the ecosystem has the combined Stream/AsyncIterator trait instead, with a single poll_next method that can signal either kind of suspension.

So if this RFC were to cover this at all, I would expect it to lean toward supporting multi-color blocks, but with the ordering of the colors being semantically irrelevant. (Indeed this is essentially what makes "algebraic effects" "algebraic.")

[rpjohnst]

@nikomatsakis

Basically, doing those transformations in an omnipresent way.

Putting the colors/effects in the type system prevents this from being an omnipresent thing. Koka uses a selective CPS transform that only touches effectful functions; Rust equivalently only applies the state machine transformation to async (and eventually gen) functions.

My point is really that "effect" need not imply anything so deep about the language that it does not already apply to Rust, and specifically that Koka's use of CPS is not really relevant to how Rust might use the term.

[clarfonthey]

This is a small nitpick, but since this is kind of a theory-heavy RFC, it felt worth pointing out that:

(Indeed this is essentially what makes "algebraic effects" "algebraic.")

…is not true. Things can be algebraic and also order-dependent, and so it's not immediately obvious why we should adopt an order-independent approach. Sure, the example you gave was reasonable, but it's unclear whether all future versions of effects/colours will fit those descriptions.

[nikomatsakis]

@rpjohnst

My point is really that "effect" need not imply anything so deep about the language that it does not already apply to Rust, and specifically that Koka's use of CPS is not really relevant to how Rust might use the term.

OK, I see. I stand corrected. Cool! I will weaken my FAQ answer later today. =)

I still prefer 'color' as the term over 'effect', but I guess I would say...I am persuadable. I agree that color is itself jargon but it strikes me as far more approachable and evocative jargon than effect. I think there's a reason the blog post was called "what color is your function" and not "what effects does your function have".

[nikomatsakis]

@Diggsey

This is... a lot to take in.

I think it is a lot less to take in than it may appear. I view this RFC as documenting existing patterns and "rounding them out" more than it is creating new ones. The only brand new thing in this RFC is committing to some syntax like async<T> or async -> T (similar to what is proposed in #3628). The rest is already part of accepted RFCs, e.g. RFC #3668 for async closures proposed the async Fn syntax. This RFC just says "we'll use the same syntax for future async traits and for const trait".

For example AsyncRead is already defined by both tokio and futures crates, and neither of them is a mechanical translation of the non-async Read trait

The RFC is not proposing a mechanical translation. async Fn is not a mechanical translation of the Fn trait. What we are saying is that the async keyword can be used as a prefix to identify translations of some kind, not that they are mechanical.

Does that make sense, @Diggsey ?

(I plan to create a FAQ from this conversation once we reach a fix point.)

[GoldsteinE]

I feel like async Read is the confusing part. Fn is already a deeply magical trait, having a magic translation to another magical trait is not that weird. Read, on the other hand, is currently not magical at all. Having async Read implies either making Read magical, or providing a way for a user to specify an “async version” of a trait, and it’s not clear how that would work.

I think that “AsyncRead should get a magical syntax if we ever add it to std” is the most novel part of this RFC, not async<T>.

[nikomatsakis]

OK, I want to engage more deeply on the naming question. It's important and I don't think color is necessarily optimal. Here are some thoughts.

When it comes to names, I think there are several qualities that the ideal name ought to have:

• Memorable and specific: A term that is too common and generic can be confusing. This can be because the word is used too much for too many things or because the word is just such a common English word that it doesn't sound like a specific thing.
• Grammatically flexible: It's good to have a term that can be used in multiple ways, for example as an adjective, noun, etc.
• Approachable: When I say "jargon" what I often mean is that it's a word that kind of has a formidable feeling, like "monad". When you hear the term you instantly feel a little bit excluded and less smart (or, alternatively, when you know the term well and use it correctly, you feel a bit smarter). That seems like a problem to me.
• Familiar: it's good to use a term that people find familiar and that gives them the right intuitions, even if it's not aligned on every detail.

There have been three names proposed, two on this thread, and one by @Nadrieril on Zulip: effect, color, and flavor.

Looking at these, I analyze them as follows.

Effect I think is reasonably memorable and specific, and it is the term I started with. I adopted color because (a) I have not found effect to be familiar nor approachable to people in conversation but also (b) it is not grammatically flexible. For example, the RFC frequently talks about K-colored traits, but it's not clear to me what the adjective form of an effect is (K-effected traits?). When I was using effect, I wrote K-traits, which I think is telling, since I had to drop the word.

Color is better but I see some downsides. First, it is such a common word in conversation that I can imagine it not being obvious it's a "term of art" here. It would be very hard to ever make it a keyword, if we found a reason to do so. Second, although I frequently see color used in the sense I mean it in informal conversation (typically referencing the "What color is your function" blog post), I wouldn't say it's familiar exactly. Third, color is relatively grammatically flexible, but I still found it not ideal. I wanted sometimes to talk about a specific "version" of a trait, and saying "the async color of the Trait" didn't sound right. There was also the problem that, as y'all are hopefully aware, the term "colored" has been associated with some awful parts of human history, and I found myself being careful about how I used the word color when writing the RFC to try and avoid evoking those connotations. All in all, not ideal for a word that may wind up being used a lot.

So this morning I've been pondering @Nadrieril's suggestion of flavor. It has all the advantages of color (familiar, memorable, approachable) but it seems to me to be actually more specific than color (I can imagine it becoming a keyword someday, for example, without quite the level of pain that color would have). It is also more grammatically flexible: we can talk about the flavor of a trait and that sounds very good, and it has no negative connotations. So I'm strongly considering switching the RFC to adopt flavor.

I'd be curious to hear a similar argument made in favor of effect. It's worth listing out the ways we'd like to use the term. I'll give examples alternating through the proposals and (typically) quoting from the RFC:

• As a name for the pattern ("a larger syntactic design pattern called the Z pattern")
• As a generic noun ("each color is tied to a specific keyword K")
• As a proper noun ("the async effect")
• As an adjective ("K-colored trait")
• As a "version" of something (e.g., "the async flavor of Fn")

It's worth pointing out that whatever word we choose, it's a 2-way door. This is a name for a design pattern that we use to maintain internal consistency for the language. If we find that the name doesn't work, we can change it, and the only thing that becomes outdated is our internal design docs and blog posts. That said, I do think the word will wind up "leaking out" into how people talk about Rust, so it's worth investing some thought into it and trying to get it right the first time.

[nikomatsakis]

@rpjohnst

I'm thinking more about this...

My point is really that "effect" need not imply anything so deep about the language that it does not already apply to Rust, and specifically that Koka's use of CPS is not really relevant to how Rust might use the term.

I agree with you here but I still find there is something that's bugging me. I think it's this. I like being able to think of "effect" as a shorthand for "side effect". But if one of the elements of a "side effect" is that it translates your function into a coroutine -- and potentially introduces a new way for the function to take input from the outside, as in the most general case of yield returning a value -- that seems to me to go beyond being a side effect of the function, and rather a different thing altogether.

So I agree that I was overrotating on the continuation passing style implementation detail, but I think I still see value in limiting the term "effect" to "side effects".

That said, if we only think of generators emitting values (and not getting values back in as input), I can see it as a generalization of side effect that works. It just seems to go beyond my intuitions, but I could get used to it. It does explain why yield_all is a better fit as the "do" operation for a generator.

(UPDATE: I guess you can think of read as a side-effect......if you map side-effect to "any interaction with your environment" essentially....)

[nikomatsakis]

Responding to my response to @Diggsey, @GoldsteinE wrote...

I feel like async Read is the confusing part. Fn is already a deeply magical trait, having a magic translation to another magical trait is not that weird.

Ah, I see-- yes, I get that. I agree that the Fn trait already has a lot of supporting syntax (sugar for (), closures, etc) and so having more special-case sugar doesn't seem strange.

I think the point of the RFC is not that async Read would be a magical trait, but that there would be some way to define async-colored traits (async flavors of traits?). Exactly what that way will be is not defined. It might be as simple as letting you have two otherwise unrelated trait definitions...

trait Read {
   fn read();
}
trait async Read {
    fn poll_read(self: Pin<&mut Self>);
    fn read() -> async -> () {
        /* something defined in terms of `poll_read` */
    }
}

...or it can be some way to automatically create async-color traits from regular ones, or it could be a mix of those things. It might also be different between colors. None of that is specified in the RFC.

What the RFC is saying is that we aim for K Trait to be a bog-standard syntax that you are used to seeing on various traits, not some piece of special sugar specific to Fn.

In any case, I agree with you that this is a new thing in the RFC (and I tried to highlight it as such).

[GoldsteinE]

It is not obvious that having trait async Read is desirable. There’re multiple possible ways to implement AsyncRead, and having one “blessed” by the standard library like that would make other implementations (for example, provided by the async runtime) second-class in comparison. There’s a chance to get into a situation where libraries would need to have documentation like “there’s async Read syntax, which you should never use, and instead use our_rt::io::AsyncRead; and use that”.

Unless, of course, async Read is imported separately and you can just use our_rt::io::{async Read}.

I think even the ability to have async Read is a huge proposal that would need its own rationale and its own consideration. This RFC currently sets “having async Read if we have async version of Read” as a goal, and I don’t think it motivates that enough, explains it enough or considers the drawbacks of this decision.

[GoldsteinE]

What the RFC is saying is that we aim for K Trait to be a bog-standard syntax that you are used to seeing on various traits, not some piece of special sugar specific to Fn.

I think it’s pretty uncontroversial for const (and maybe unsafe?), but has huge implications and trade-offs for async, which are not explored in the RFC.

[Diggsey]

I think the point of the RFC is not that async Read would be a magical trait, but that there would be some way to define async-colored traits (async flavors of traits?). Exactly what that way will be is not defined. It might be as simple as letting you have two otherwise unrelated trait definitions...

Ok, that does explain it better. It does raise some concerns about namespacing though. If dyn Read could be a completely different trait from Read, how would I import it? Do I use std::io::dyn Read? That seems problematic. If they all share the same qualified name, then what if I want to use a different dyn Read, but still use the original Read? Who gets to define a dyn Read - do we need a new set of orphan rules?

[nikomatsakis]

@Diggsey You would import Read, yes, and you would write dyn async Read, just as (under RFC #3668) you write dyn async Fn().

If they all share the same qualified name, then what if I want to use a different dyn Read, but still use the original Read? Who gets to define a dyn Read - do we need a new set of orphan rules?

If you write dyn Read, you are getting the synchronous color of Read. If you write dyn async Read, you are getting the asynchronous version. The orphan rules would be the same: any place you can write an impl for Read, you can also write one for async Read.

In short, you can think of a trait like Fn (or Read) as having a default, uncolored version plus some set of K-colored alternatives (not every alternative is necessarily available for every trait). You select the colored version with the keyword.

(Bear in mind that all of this is going beyond what's specified in the RFC, which specifically avoids saying what async means when applied to any trait beyond Fn, and these questions don't arise in that particular case because Fn itself is unstable and cannot presently be implemented.)

[GoldsteinE]

Bear in mind that all of this is going beyond what's specified in the RFC, which specifically avoids saying what async means when applied to any trait beyond Fn

I guess it’s just weird that it says that we should have it mean something, without exploring whether there is a meaning that makes sense.

[yoshuawuyts]

@nikomatsakis said:

But if one of the elements of a "side effect" is that it translates your function into a coroutine -- and potentially introduces a new way for the function to take input from the outside, as in the most general case of yield returning a value -- that seems to me to go beyond being a side effect of the function, and rather a different thing altogether.

To maybe help frame this: Koka differentiates between effect types and effect handlers. Effect handlers are sometimes also referred to as typed continuations. Effect handlers in Koka are expressed in the type system as effect types. But not all effect types are also effect handlers. An example of an effect type in Koka which is not an effect handler is divergence (div in Koka).

Divergent functions in Koka are functions which do not statically guarantee they will terminate. This cannot be modeled in terms of coroutines, despite having runtime implications. Instead it directly represents a language-level capability. Daan Leijen (Koka's lead) explained this as follows:

The effect types are not just syntactic labels but they have a deep semantic connection to the program [...]. For example, we can prove that if an expression that can be typed without an exn effect, then it will never throw an unhandled exception; or if an expression can be typed without a div effect, then it always terminates

This is why I tend to think of effect types more like language-level capabilities or permissions. With typed continuations (effect handlers) representing a special subset of those which map to e.g. async/.await, gen/yield_all, and try/?.

[nikomatsakis]

@GoldsteinE I appreciate you raising the point and helping me to understand it. I'm going to be writing up a summary for an FAQ and bringing it to the broader lang team as part of our review from this thread. I'll drop a note here with a link so you can double check I've accurately reflected your opinion. (Same for you, @clarfonthey)

[ssokolow]

multi-colored blocks does not seem worthwhile. K9 { K7 { K8 { $expr } } } is clearer than K4 K6 K5 { $expr } in emphasizing the application order.

However, given current rustfmt convention, that would get rendered as this:

K9 {
    K7 {
        K8 {
            $expr
        }
    }
}

...which I don't find reasonable. I don't think it's a good idea to encourage either more #[rustfmt::skip] or a glaring inconsistency in how rustfmt handles blocks... especially when you may have to bump the #[rustfmt::skip] up in the token tree to avoid a "can't use that there" error.

[tmandry]

For async Read/Write traits, I consider the current state-of-the-art to be this proposal by @nrc. It proposes generic Read and Write traits that look like async versions of the existing traits, plus specialized "Ready" and "Owned" subtraits of each that specialize for readiness and completion-based I/O respectively.

The Read and Write traits in the proposal would be called async std::io::Read and async std::io::Write under this RFC. It's not crystal clear to me if ReadyRead and OwnedRead would be async ReadyRead and async OwnedRead or if those traits are async-only. I think they both have obvious equivalent blocking semantics, but I can only imagine wanting those semantics in the Ready case, not the Owned case.

This RFC doesn't need to answer all of those questions, but I think it's important to consider that some traits will have sync and async versions while some will only have one or the other. For some, it may not be clear at the time of stabilizing whether we will want both. This only complicates naming under this RFC if it's an async-first trait. I feel pretty confident that we can come up with a way of resolving such uncertainties, both because I think it should be pretty clear by the time of stabilizing whether we will want both, and because for any traits where it's not completely clear, we can commit to stabilizing an explicit async flavor.

[clarfonthey]

multi-colored blocks does not seem worthwhile. K9 { K7 { K8 { $expr } } } is clearer than K4 K6 K5 { $expr } in emphasizing the application order.

However, given current rustfmt convention, that would get rendered as this:

K9 {
    K7 {
        K8 {
            $expr
        }
    }
}

...which I don't find reasonable. I don't think it's a good idea to encourage either more #[rustfmt::skip] or a glaring inconsistency in how rustfmt handles blocks... especially when you may have to bump the #[rustfmt::skip] up in the token tree to avoid a "can't use that there" error.

I mean, given how we have a predefined ordering for the keywords, it would make senses that, for example:

const {
    unsafe {
        expr()
    }
}

could be shortened to:

const unsafe {
    expr()
}

if needed.

[PeterHatch]

So I disagree with pretty much all the reasons for not going with AsyncFn, and wanted to go over why, for each of them.

First, the story of how to transition something from sync to async gets more complicated. It's not a story of "just add async keywords in the right places".

I think that story is only simpler at that high a level of abstraction. If you know what the right places are, you know why they need to change. If you know you want a different version of a trait, changing the name is what you do every other time that's what you need, so that seems simpler to me. If you don't know you want a different version of a trait, just adding a keyword means you still may not understand what the change did, where changing the name is extremely clear.

Also, even if that's the story we want, it's not going to be the case for any traits outside the standard library. Consistency across the ecosystem seems important; if we eventually give the rest of the ecosystem the tools to change and encourage them to do so, we can change the standard library then as well.

Second, this convention does not offer an obvious way to support const traits like const Default, unless we are going to produce ConstDefault variants as well somehow.

Absolutely right. AsyncTrait is the obvious solution for async traits, and doesn't work for const. I don't think that means we need some other consistent method, I think it means those are fundamentally different things and treating them as the same is a false equivalency that would make the language harder to learn.

And if there are more variants in the future, e.g., AsyncSendSomeTrait or AsyncTrySendSomeTrait, it becomes very unwieldy.

Keywords would also become unwieldy in this case. Only differences I can see is that keywords might not require a specific order, and that long names benefit from auto-completion and the ability to import as a shorter name.

Third, although we are not committing to any form of "flavor generics" in this RFC, we would also prefer not to close the door entirely; using an entirely distinct identifier like AsyncFn would make it very difficult to imagine how one function definition could be made generic over a flavor.

You just have AsyncFn be an alias for the generic version with the async parameter true. It's not at all clear to me that that wouldn't be desirable in general - being generic over async is extra complexity that is pretty often unwanted. My biggest concern with the idea of flavor generics is that they could make learning the language, and the standard library in particular, more complicated for everyone. I don't think committing to the extra complexity in the standard library even before we've committed to flavor generics is the way to go.

For the last point, and just in general, I consider keywords to be quite complicated syntax; since they can be used for pretty much anything, you have to specifically learn what they mean in each case. If there's a question of using a keyword or some other method, I lean towards the other method.

[nikomatsakis]

The lang team will be discussing this RFC in a design meeting -- I think today but I might be wrong -- and so I prepared a summary of the discussion here:

https://hackmd.io/w2-etPU7RouzxJ6GJ1fbSg

Feedback welcome. @PeterHatch I hadn't seen your comment yet so I'm going to read it over and make sure it's reflected in there. Some sections to note...

• The syntax async Read implies a "primary" read trait for async
• Choice of name

I also took another stab at the RFC summary and framing that I'd be curious to get people's read on (is it clearer?):

The primary goal of this RFC is to commit to using async Fn as the syntax for async closures (as proposed in [RFC #3668][]) and to adding some form of syntax for 'async flavored types' similar to what is proposed in [RFC #3628][]. This "async flavored type" notation, currently denoted with the placeholder syntax 🏠async<$ty>, would be equivalent to impl Future<Output = T> (and hence its precise meaning would be dependenent on where it appears, just as impl Future can expand in different ways).

The motivation for comitting to both async Fn and 🏠async<$ty> syntaxes together is to create a coherent design pattern around async that allows the user to understand all aspects of the "async flavor" of Rust as a unit. The goal is that taking sync code and making it async can be done by adding "async" and "await" in the appropriate places without hitting places where other abstractions "leak through".

The second goal of this RFC is to make strong suggestions for the syntax of future features that are under consideration. Based on async, the RFC defines a flavor design pattern. The flavor design pattern is a series of syntax recommendations and transformations that can be applied to any "flavor keyword" K. A flavor keyword K is some keywordthat can be applied to a function or a block, like K fn foo() or K { /* something */ }, and which has the "infectious" property, meaning that code with flavor K interacts naturally with other code of the same flavor, but only in limited ways with code of other flavors. Beyond async, other examples of existing flavor keywords are const and unsafe. (There are also other "flavor-like" things in Rust that do not have keywords, like functions that operate on &T/&mut T/T; these are not described by this RFC directly.)

Based on the flavor design pattern, the RFC recommends that we use const $Trait as the syntax to indicate a version of $Trait in which some or all members are made const-flavored and that we use async $Trait as the syntax for future "async flavors" of traits that we may decide to add (e.g., async Read). The RFC does NOT define any such traits nor propose any kind of means to define them, mechanical or otherwise; the only criteria is that K $Trait should have a superet of the members of $Trait but where some have been made K-flavored (async Fn as defined in [RFC #3668] meets this definition). The RFC includes a "future possibilities" section that describe various things we could do, such as what const $Trait might mean, how unsafe Trait could work (and might not), and the possibility of "flavor generics" (aka, effect generics or keyword generics). None of those features are proposed in this RFC.

Please feel free to leave hackmd comments in the doc.

[PeterHatch]

So, to add context to my earlier comments, somewhat discussed under "Non-mechanical mechanisms" in the HackMD - I think the issue was that I wasn't clear about the steps that got me to my opinion, and assumed they were shared. To be specific, I think:

1. If we're committing to a syntax for consistency reasons, that consistency needs to be shared across the whole Rust ecosystem, not just the standard library, so it needs a mechanical way to be specified in Rust code.
2. Given 1, and that we're not committing to flavor generics, we need to know there is some other way to accomplish the task.
3. And after those two points, we get to the actual argument I tried to make, that this design space isn't explored enough for us to commit to it. I could continue arguing this point, but I'm not sure this is actually the source of any disagreement.

[rpjohnst]

On the section of the meeting notes about try and Iterator/Gen: the distinction you're running into here is that Iterator (or Gen) is already itself a "flavor carrier," like Future, and unlike e.g. Read/Default/Clone/Into.

It doesn't make sense to apply another flavor to a flavor-carrying trait, because it mixes registers or levels of abstraction. This is the exact same thing that has been thoroughly discussed in terms of streams. It is also something that RFC #3628 might address- you can freely add/remove flavors to something like try gen T, without burdening the flavor carriers themselves with this awkward layer-mixing, because the try gen T syntax can lower to the appropriate multi-flavor carrying trait instead.

(Edit, reading further: To be clear, this is not unique to async gen. Fundamentally any combination of early-exit-with-possible-resume flavors has this property. There should be no difference between try async and async try, because the function/block itself will always be able to freely intermix ?s and .awaits. If you do actually mean to have a return type like Result<impl Future>, that can be obtained without any interaction with this flavor-combination system.)

[Darksonn]

For async Read/Write traits, I consider the current state-of-the-art to be this proposal by @nrc. It proposes generic Read and Write traits that look like async versions of the existing traits, plus specialized "Ready" and "Owned" subtraits of each that specialize for readiness and completion-based I/O respectively.

Sorry for being late to this discussion. I just saw these traits, and I think they're way inferior to the current poll-based designs used by Tokio or futures. The problem is that you're making async Read and async Write incompatible with each other. If read takes a mutable reference to self, then you cannot perform any writes while the read is ongoing, and vice-versa. Not being able to read and write at the same time is unworkable.

The way this is worked around in std is by implementing Read and Write on shared references, so we have &TcpStream and &File being readable. The problem is that you wanted to go from "1 reader or 1 writer" to "1 reader and 1 writer", but you ended up at "N readers and N writers". Implementing IO resources that support "N readers" or "N writers" is way more difficult than supporting "1 reader and 1 writer". It's so bad that the only correct implementation I've ever seen of this is the one in Tokio. The only way to do it without allocating memory involves intrusive linked lists.

Now, you might argue that the Ready trait does support concurrent reads and writes. Well, for one, you just gave up support for the completion based traits. But there's a more important issue: cancellation safety. Using Ready to perform concurrent reads and writes absolutely requires that you use it with tokio::select! or equivalent and that you sometimes cancel the ready future. This makes manually implementing the Ready trait really difficult since you have to be careful to write cancel safe code.

Even if you add "ready must be cancel safe" to the contract of Ready, there's yet another issue: Performing IO with a combined ready method that combines waiting for both reading and writing into a single function is even more inconvenient than writing poll_* functions. You basically are forced to architect your code around a massive tokio::select!. To see this for yourself, I challenge you to write a correct copy_bidirectional using these traits.

Edit: This is probably off-topic. Please reply to this comment here instead.

[camsteffen]

I have some concerns about introducing 🏠K<$ty>.

To understand a flavor is to understand its output type. I can't really explain a try block without telling you about Result, or gen without Iterator, or async without Future. This is demonstrated in the "teaching" section of the RFC where you define what a future is in the first few sentences of teaching async functions. So I don't think that hiding away that type with another syntax helps with learnability of the feature, because learning that specific type is critically important to understanding the flavor. If anything, naming that type in the code is helpful.

For devs of any experience level, how are we going to pronounce async<T>? I'd have to say "a future of T". The syntax would just necessitate some mental translation to read it.

Is there even one other possible flavor where that syntax might work?

All the other parts of the language that can be made consistent makes sense to me, but the types really are special and should not be over-abstracted IMO.

[traviscross]

One thing that may not be immediately apparent is that the stabilization of RFC 3668 async closures had become blocked on this RFC.

For that feature, we have to choose, as the bounds syntax, between async Fn{,Mut,Once} and AsyncFn{,Mut,Once}. The RFC had left this as an open question.

We had agreed, in later lang discussion, that we might have consensus on async Fn as that syntax if and only if we had agreement that we would generalize this somehow to other traits. This RFC is the outcome of that.

On the plus side, there are many valuable bits in this RFC, and I hope that we are able to make something of those later. I'm glad that @nikomatsakis wrote this up, and working through this has I think sharpened all of our thinking. The resulting discussion on this thread and in Zulip has been particularly illuminating, and many people have made a number of compelling points.

But overall, I've come to believe that this RFC is trying to do too much and is foreshadowing too much when we have too many question marks. It's proposing a big model, and I'm just not sure that we've really nailed it.

As one concrete matter, while I was supportive of async Fn, I've realized more clearly that my reasons for that do not necessarily generalize to other traits. The Fn traits are special in many ways, and in fact, this specialness is what had originally prompted @compiler-errors to suggest that the async Fn syntax could be adopted without having to decide generally about async traits at all.

As has been discussed on this thread, where things get particularly difficult is when the trait that would be affected represents the state machine for some other "K". I remain strongly skeptical that we would ever want to have K OtherKTrait in these instances.

Looking beyond that, I sense that it's harder to apply this K Trait idea whenever the trait represents any kind of nontrivial finite state machine, and I suspect that is part of why we have so many question marks about the combination of this with traits like Read.

At the same time, I believe that we will eventually have a first class "K-style" notion in the language. I'm just not confident that notion necessarily implies K Trait. For traits, it seems at least possible that we will just have a handful of special cases (e.g. for Fn, Default, Clone, etc.) or that we may have other available approaches for solving the relevant problems.

In that light, I particularly want to decouple async closures from this RFC and allow those to move forward. Personally, I've warmed to shipping async closures with AsyncFn{,Mut,Once} bounds syntax, and I know from discussion that some other members of the team feel positively about this also. The author of RFC 3668, @compiler-errors, has also warmed to this.

In support of this, @nikomatsakis has pointed out in discussion that, since we wouldn't be soon shipping any kind of generalized form that would allow the ecosystem to write async Foo, we are and will continue seeing AsyncFoo traits appear regardless. And that if we did later come up with some generalized mechanism, we could at that time manage the transition from AsyncFn to async Fn.

I agree with and am persuaded by that also. I believe it's likely at this point that we will decouple async closures from this RFC and proceed to propose they be stabilized using the AsyncFn bounds syntax.

As mentioned, there are many bits of this RFC that I do find valuable, and I hope that we do come back separately to these and are able to benefit from the careful thinking that went into this document and into the discussions that have been prompted by it.

[PeterHatch]

From the meeting notes:

I do find the idea that you can import one name and add flavors to it later quite compelling.

Note that needing to import a trait manually helps avoid new ambiguities being added to code due to changes in other crates. Opting out of that seems like it could cause problems.

I find it kind of annoying that you have to go import TryFrom everytime you want to use that instead of regular From, for example.

If I'm understanding correctly, under this proposal flavored traits are supposed to have the same members as the base trait, meaning try From would have a from method instead of try_from; so for any struct that implements both From and try From, I think you'd have to disambiguate which trait you were using every time? That seems much worse than needing to import TryFrom.

[nikomatsakis]

So, building on what @traviscross said, I'm inclined to close the RFC. I definitely feel disappointed, as writing the RFC helped clarify many things and I liked the idea of reducing the uncertainty a bit. But I also think it will be just fine to stabilize AsyncFn and continue our discussions.

Like TC, I ultimately do believe we should have some form of K-generics -- and I mean that in a fuller sense than this RFC, I think it's important that we are able to write combinator-like APIs (e.g., Iterator) that work whether or not you have ?, await, const, etc. Unlike TC, I am 100% confident that part of this will be "K-flavored" traits.

However, I don't think we need to stabilize async Fn syntax now to get there, and I would really like to see async Fn stabilized ASAP. I also feel that we should not use async Fn unless we are prepared to apply the async keyword to other traits, it will just be confusing for no reason. It seems better to start with AsyncFn -- if we want, we can always make that a (deprecated) trait alias in the future.

[rpjohnst]

I don't think this (discussion about more general "K-flavoring") really changes anything about the idea that the Fn trait sugar was designed to match fn item syntax.

More specifically: regardless of whether we start with AsyncFn or async Fn, we are extending that sugar. I would thus argue for async Fn over AsyncFn even if we never added async to any other traits. And this line of discussion was already covered pretty heavily in the async closures RFC.

[Diggsey]

I think this is the right choice - it will be much easier to discuss this idea once we have a more concrete idea of how the syntax would be used in practice.