- Where: zoom.us (see Registration below)
- When: February 9, 17:00-18:00 UTC
- Contact:
- Name: Lin Clark
- Email: [email protected]
If this is your first time attending, please fill out the registration form to receive an invite.
The meeting is open to CG members only. You can join the CG here.
The meeting will be on a zoom.us video conference.
- Opening, welcome and roll call
- Please help add your name to the meeting notes.
- Please help take notes.
- Thanks!
- Announcements
- Submit a PR to add your announcement here
- Proposals and discussions
- A possible roadmap for WASI Preview1, Preview2, Preview3, and WASI 1.0 - Dan Gohman (slides: pdf)
- A proposed framework for Preview3 and WASI 1.0: WebAssembly/WASI#515
- Reorganizing the Preview1 files: WebAssembly/WASI#510
- Should Preview1
fd_readdirfilter out.and..?
- Should we start a new WASI proposal repo for a CLI world?
- Lin Clark
- Joel Dice
- Bailey Hayes
- Slava Kuzmich
- Steve Schoettler
- Pat Hickey
- Zalim Bashorov
- Jeff Charles
- Dave Hay
- Hung-Ying Tai
- Adam Mohammed
- Dan Gohman
- Piotr Sikora
- Ashley Nelson
- Chinmay Garde
- Ivan Font
- Frank Schaffa
- Antoni Bofarull
- Yong He
- Sam Clegg
- Till Schneidereit
- Colin Murphy
- Andrew Brown
- David Justice
- Kyle Brown
- Deepti Gandluri
Dan Gohman: For quite some time no clear path where it would be. New sense of clarity with wit tooling in place. Proposal for thinking through scope of each step on the path
Preview 1 is what’s out there today, supported in many engines, some prod envs. Focus: let’s support existing users. PR 510 is tidying up preview 1 docs to clarify existing semantics. Compat with what we have now is the theme.
Preview 2 is the big item, based on wit idl. That gives us answers to longstanding questions. Things like modularization. Interfaces and worlds are concepts that WASI can build on there. Things like what is a file descriptor, etc. Never had clear answers, and now the component model gives us that. What should be WASI’s job vs low level standards? Now emerging that there are clear ways to separate. Other layers can define those pieces and WASI can be pure API standardization. How to make WASI interfaces fully virtualizable. With CM, we have answer for that. We now have answers for thses things, moment of clarity. Number of lessons learned in Preview 1 fixed in Preview 2.
Preview 3. In Preview 2 timeframe won’t be able to do integrated async. Points to Preview 3 being where we add async features. This adds future and stream of T. Bindings for different languages is pretty complex. Check out Luke’s proposal. Big benefit of integrated async, can have single global event loop for composition (even though there may be nested event loops), idiomatic source language bindings, and ability to have typed streams. Turn async up to 11
Beyond that, that feels like the last big platform feature for the foundation. Big push for WASI 1.0 is standardization and process. Starting to work with WASM CG, getting an editor, publishing the document. WASI now has this concept of worlds. At point of 1.0, we’ll have some worlds. WASI 1.0 will just include those worlds and interfaces that are standardized at that time.
Preview 1: Support existing users Preview 2: Rebase WASI on wit Preview 3: Level up async 1.0: Standardize
At each point, can incorp real world feedback. May need to make breaking changes. Preview 2 -> 3 and 3->1.0 should be easier than the transition from Preview 1->2. Not going to promise Preview 2 forever, but these transitions won’t fundamentally change what we need to do.
Questions?
Andrew Brown: Which proposals do you see as being a part of Preview 2, Preview 3, and 1.0. Do you see just the 5 original, or can several other of the 20 be included.
Dan Gohman: At each stage, we’ll say everything that is ready can join the party. Just about timing. Timeline defined by CM and wit tooling readiness. Original fs, clocks, random set. Hoping those will be ready, Hoping that other proposals could advance and be ready.
Andrew Brown: So you know what’s required what’s needed for Preview 2, but other authors don’t necessarily have that insight.
Dan Gohman: Good question. Next step for me to write up docs. Then we can put together a framework, and as part of that explain that you need to write in the wit format. Could add other criteria, but the big criteria is wit. For Preview 3, if async is relevant, you make the changes, but otherwise no changes. For WASI 1.0, would need to go through CG.
Bailey Hayes: Thanks for putting this together. I’d like to see—we want WASI to be modular. Some don’t make sense for all runtimes. I think one thing that will help is showing that now with the modular set of interfaces what it looks like when one isn’t supported.
Dan Gohman: Great idea. I think the world concept is the first building block to getting there. That will define sets of APIs. The world is the nexus between developers and platforms. WASI subgroup will define some number of standardized worlds. Other worlds people have talked about, e.g. embedded systems. You can also build worlds on top of worlds with your custom APIs.
CW: Exactly what Bailey said, thanks for this hard work. From an external perspective, WASI user C developer point of view. What’s the impact. Preview 1, not much. Preview 2, would need C interfaces going towards wit. For P2, have a situation where I could create a component with P2, but it will only execute on what supports that. P3 is going to be the same.
Dan Gohman: Yes, there will be a transition period. One thing that is happening is a preview 1 -> 2 adapter toolchain. Can take Preview 1 impl forward migration
CW: Then engine impl will just have to play catch up. We may be able to help with C++ runtimes. If you could keep us in mind.
Dan Gohman: WASI sg meeting is a great place to keep up with things. Also follow me on social media, Mastodon
Piotr Sikora: Especially like that you aren’t abandoning Preview 1 users. Question, what timeline? Especially if we want feedback, sounds like multi-year process before WASI 1.0.
Dan Gohman: That’s right, don’t expect to be as long as Preview 1.
Piotr Sikora: Are we expecting at least 1 year between?
Dan Gohman: For Preview 3 we’re talking about moving once the tooling is ready. ONce we have the tooling that can do it, I think we’ll want to make that move. A year might not be a bad place to start in terms of timelines.
Piotr Sikora: Perfect, thanks. Between p1 and p2 main feature is moving to wit. That’s mostly about how interfaces are generated. Do we expect those to change at the ABI level
Dan Gohman: ABI does change. P2 is using CM, using canonical ABI
Piotr Sikora: So some of the proposals including WASI HTTP are blocked on async features.
Luke Wagner: Dan currently has WASI I/O with psuedofutures and pseudostreams. This is the stop gap. The idea is to publish a P2 version that has these pseudofutures and streams. This won’t give us the magic composability, but fine for now.
Piotr Sikora: Perfect, thank you
Till Schneidereit: Big fan of this roadmap. One thing I’m curious about is how much you’ve been thinking about what degree the different milestones should be covered by what level of test coverage for APIs. Besides tests, what about more substantial description of interfaces. Obviously, for WASI 1.0, we’ll need to have a solid interop test suite. Do you think that’s something that should happen for P2 and P3 as well, or will that need to join at 1.0.
Dan Gohman: I do want to shout out to WASI test suite. Right now being built on P1. P1 spec is not very complete, so this is helping flush out. Hopeful that these can be upgraded to P2 because these are being written as source language tests rather than raw wat. Don’t have strong opinion about whether we should block on tests. A little tricky with some of the APIs. Not clear that we’d want to block Preview 2 on whole testing framework. For command style APIs is that they are really easy to test. Once we get outside that space, harder to write tests. Don’t know what our answer should be: should we require for advancement? Would want group to decide
Till Schneidereit: I do feel like we should require some kind of test suite at a certain stage because otherwise hard to see how to have interop implementation. Maybe equivalent to Web Driver. For now, neither here nor there. Was just curious about your current thinking.
Dan Gohman: My personal thinking on that is letting each proposal figure out its own path. Not ideal in some respects, but does give us flexibility to allow for specific needs of APIs. Similar to current story on portability. Different APIs can proposal limit sets of portability. E.g. EBPF probably not portable to Windows. My thinking is we approach testing in the same way. Probably not the worst situation to be in. Progress here probably happens iteratively over time.
Deepti Gandaluri: You were talking about portability already. Some amount of portability that P1->P2 and P2->P3 provide guarantees?
Dan Gohman: Current thinking is we continue with current scheme will define its own thing. All the command APIs expect to support Windows, MacOS, Linux. Other APIs can define what they do. WASI as a whole with the framework we’re talking about, lots won’t be portable. We don’t want to preclude those to not being part of WASI. Maybe the world mechanism is a way to impose more portability rules. Feels to me we can do it world by world.
Deepti Gandaluri: Think that answers my question. Mentions that at each level would be taking implementation feedback. Maybe the worlds answer this to an extent, but what impl feedback is what you block on or move to a further preview.
Dan Gohman: This is more open ended because there’s more diversity between engine impl styles. Same thing basically applies though. If someone raises a problem for core, like SIMD, and one person says that it doesn’t work on x86, that applies to most engines. We can use the same process of figuring out how valid that concern is. In the core wasm spec process, we just work through that and champion makes judgement call. Those are the decision making process that we follow. We can say we trust the champion. If someone has some super exotic changes, then the champion can define the scope. E.g. someone trying to impl a fs on a system that doesn’t have an fs, the champion can scope to only platforms that have fs and figure out the others with a different proposal.
Currently champions can make decisions, CG can vote.
Deepti Gandaluri: We do have a minimal subset of two web vms
Dan Gohman: We do have an analog for that, where the SG votes on each proposals
Deepti Gandaluri: I see the parallel to that, but was harder for me to reason about
Luke Wagner: I think some of the intention between the WebVMs is that you want impl feedback from people who have a real commitment to longterm security, maintainability. That meant sign off meant something. That was 4-5 years ago. In the current state, we do have long term thinking outside the browser so I think we’ll want to find out some verbiage to capture that long term thinking.
Deepti Gandaluri: Probably something we’ll want to define for core too.
Dan Gohman: Now that wit can have worlds, we can use existing proposal template to define worlds. That could just be a WASI level proposal. Should we define a new proposal template for a world.
Luke Wagner: From my perspective, makes sense to have repo as world. Possibly have worlds and itnerfaces separated by worlds. Other case, couple HTTP interface with HTTP proxy world. Don’t want to always separate them, may have collection of interfaces and worlds in the same repo.
Dan Gohman: Makes sense, I’ll have a new proposal for our next meeting
Dan Gohman: Just calling attention to the PR.
Dan Gohman: POSIX historically included. A lot of languages have filtered those out because users don’t care about those. Idea is that we could also filter these out and then re-add them. Then there’s a question of what Preview 1 should do. Could just continue to do the POSIX conforming. Curious about sg
Pat Hickey: Because there are existing P1 programs, we shouldn’t make changes like this.
Dan Gohman: I’m seeing sam as a thumbsup up there. More broadly, I think any changes to P1 should increase compat and we should try not to introduce non-determinism. That sound right? Looks like agreement
- Lin Clark
- Joel Dice
- Bailey Hayes
- Slava Kuzmich
- Steve Schoettler
- Pat Hickey
- Zalim Bashorov
- Jeff Charles
- Dave Hay
- Hung-Ying Tai
- Adam Mohammed
- Dan Gohman
- Piotr Sikora
- Ashley Nelson
- Chinmay Garde
- Ivan Font
- Frank Schaffa
- Antoni Bofarull
- Yong He
- Sam Clegg
- Till Schneidereit
- Colin Murphy
- Andrew Brown
- David Justice
- Kyle Brown
- Deepti Gandluri
Dan Gohman: For quite some time no clear path where it would be. New sense of clarity with wit tooling in place. Proposal for thinking through scope of each step on the path
Preview 1 is what’s out there today, supported in many engines, some prod envs. Focus: let’s support existing users. PR 510 is tidying up preview 1 docs to clarify existing semantics. Compat with what we have now is the theme.
Preview 2 is the big item, based on wit idl. That gives us answers to longstanding questions. Things like modularization. Interfaces and worlds are concepts that WASI can build on there. Things like what is a file descriptor, etc. Never had clear answers, and now the component model gives us that. What should be WASI’s job vs low level standards? Now emerging that there are clear ways to separate. Other layers can define those pieces and WASI can be pure API standardization. How to make WASI interfaces fully virtualizable. With CM, we have answer for that. We now have answers for thses things, moment of clarity. Number of lessons learned in Preview 1 fixed in Preview 2.
Preview 3. In Preview 2 timeframe won’t be able to do integrated async. Points to Preview 3 being where we add async features. This adds future and stream of T. Bindings for different languages is pretty complex. Check out Luke’s proposal. Big benefit of integrated async, can have single global event loop for composition (even though there may be nested event loops), idiomatic source language bindings, and ability to have typed streams. Turn async up to 11
Beyond that, that feels like the last big platform feature for the foundation. Big push for WASI 1.0 is standardization and process. Starting to work with WASM CG, getting an editor, publishing the document. WASI now has this concept of worlds. At point of 1.0, we’ll have some worlds. WASI 1.0 will just include those worlds and interfaces that are standardized at that time.
Preview 1: Support existing users Preview 2: Rebase WASI on wit Preview 3: Level up async 1.0: Standardize
At each point, can incorp real world feedback. May need to make breaking changes. Preview 2 -> 3 and 3->1.0 should be easier than the transition from Preview 1->2. Not going to promise Preview 2 forever, but these transitions won’t fundamentally change what we need to do.
Questions?
Andrew Brown: Which proposals do you see as being a part of Preview 2, Preview 3, and 1.0. Do you see just the 5 original, or can several other of the 20 be included.
Dan Gohman: At each stage, we’ll say everything that is ready can join the party. Just about timing. Timeline defined by CM and wit tooling readiness. Original fs, clocks, random set. Hoping those will be ready, Hoping that other proposals could advance and be ready.
Andrew Brown: So you know what’s required what’s needed for Preview 2, but other authors don’t necessarily have that insight.
Dan Gohman: Good question. Next step for me to write up docs. Then we can put together a framework, and as part of that explain that you need to write in the wit format. Could add other criteria, but the big criteria is wit. For Preview 3, if async is relevant, you make the changes, but otherwise no changes. For WASI 1.0, would need to go through CG.
Bailey Hayes: Thanks for putting this together. I’d like to see—we want WASI to be modular. Some don’t make sense for all runtimes. I think one thing that will help is showing that now with the modular set of interfaces what it looks like when one isn’t supported.
Dan Gohman: Great idea. I think the world concept is the first building block to getting there. That will define sets of APIs. The world is the nexus between developers and platforms. WASI subgroup will define some number of standardized worlds. Other worlds people have talked about, e.g. embedded systems. You can also build worlds on top of worlds with your custom APIs.
CW: Exactly what Bailey said, thanks for this hard work. From an external perspective, WASI user C developer point of view. What’s the impact. Preview 1, not much. Preview 2, would need C interfaces going towards wit. For P2, have a situation where I could create a component with P2, but it will only execute on what supports that. P3 is going to be the same.
Dan Gohman: Yes, there will be a transition period. One thing that is happening is a preview 1 -> 2 adapter toolchain. Can take Preview 1 impl forward migration
CW: Then engine impl will just have to play catch up. We may be able to help with C++ runtimes. If you could keep us in mind.
Dan Gohman: WASI sg meeting is a great place to keep up with things. Also follow me on social media, Mastodon
Piotr Sikora: Especially like that you aren’t abandoning Preview 1 users. Question, what timeline? Especially if we want feedback, sounds like multi-year process before WASI 1.0.
Dan Gohman: That’s right, don’t expect to be as long as Preview 1.
Piotr Sikora: Are we expecting at least 1 year between?
Dan Gohman: For Preview 3 we’re talking about moving once the tooling is ready. ONce we have the tooling that can do it, I think we’ll want to make that move. A year might not be a bad place to start in terms of timelines.
Piotr Sikora: Perfect, thanks. Between p1 and p2 main feature is moving to wit. That’s mostly about how interfaces are generated. Do we expect those to change at the ABI level
Dan Gohman: ABI does change. P2 is using CM, using canonical ABI
Piotr Sikora: So some of the proposals including WASI HTTP are blocked on async features.
Luke Wagner: Dan currently has WASI I/O with psuedofutures and pseudostreams. This is the stop gap. The idea is to publish a P2 version that has these pseudofutures and streams. This won’t give us the magic composability, but fine for now.
Piotr Sikora: Perfect, thank you
Till Schneidereit: Big fan of this roadmap. One thing I’m curious about is how much you’ve been thinking about what degree the different milestones should be covered by what level of test coverage for APIs. Besides tests, what about more substantial description of interfaces. Obviously, for WASI 1.0, we’ll need to have a solid interop test suite. Do you think that’s something that should happen for P2 and P3 as well, or will that need to join at 1.0.
Dan Gohman: I do want to shout out to WASI test suite. Right now being built on P1. P1 spec is not very complete, so this is helping flush out. Hopeful that these can be upgraded to P2 because these are being written as source language tests rather than raw wat. Don’t have strong opinion about whether we should block on tests. A little tricky with some of the APIs. Not clear that we’d want to block Preview 2 on whole testing framework. For command style APIs is that they are really easy to test. Once we get outside that space, harder to write tests. Don’t know what our answer should be: should we require for advancement? Would want group to decide
Till Schneidereit: I do feel like we should require some kind of test suite at a certain stage because otherwise hard to see how to have interop implementation. Maybe equivalent to Web Driver. For now, neither here nor there. Was just curious about your current thinking.
Dan Gohman: My personal thinking on that is letting each proposal figure out its own path. Not ideal in some respects, but does give us flexibility to allow for specific needs of APIs. Similar to current story on portability. Different APIs can proposal limit sets of portability. E.g. EBPF probably not portable to Windows. My thinking is we approach testing in the same way. Probably not the worst situation to be in. Progress here probably happens iteratively over time.
Deepti Gandaluri: You were talking about portability already. Some amount of portability that P1->P2 and P2->P3 provide guarantees?
Dan Gohman: Current thinking is we continue with current scheme will define its own thing. All the command APIs expect to support Windows, MacOS, Linux. Other APIs can define what they do. WASI as a whole with the framework we’re talking about, lots won’t be portable. We don’t want to preclude those to not being part of WASI. Maybe the world mechanism is a way to impose more portability rules. Feels to me we can do it world by world.
Deepti Gandaluri: Think that answers my question. Mentions that at each level would be taking implementation feedback. Maybe the worlds answer this to an extent, but what impl feedback is what you block on or move to a further preview.
Dan Gohman: This is more open ended because there’s more diversity between engine impl styles. Same thing basically applies though. If someone raises a problem for core, like SIMD, and one person says that it doesn’t work on x86, that applies to most engines. We can use the same process of figuring out how valid that concern is. In the core wasm spec process, we just work through that and champion makes judgement call. Those are the decision making process that we follow. We can say we trust the champion. If someone has some super exotic changes, then the champion can define the scope. E.g. someone trying to impl a fs on a system that doesn’t have an fs, the champion can scope to only platforms that have fs and figure out the others with a different proposal.
Currently champions can make decisions, CG can vote.
Deepti Gandaluri: We do have a minimal subset of two web vms
Dan Gohman: We do have an analog for that, where the SG votes on each proposals
Deepti Gandaluri: I see the parallel to that, but was harder for me to reason about
Luke Wagner: I think some of the intention between the WebVMs is that you want impl feedback from people who have a real commitment to longterm security, maintainability. That meant sign off meant something. That was 4-5 years ago. In the current state, we do have long term thinking outside the browser so I think we’ll want to find out some verbiage to capture that long term thinking.
Deepti Gandaluri: Probably something we’ll want to define for core too.
Dan Gohman: Now that wit can have worlds, we can use existing proposal template to define worlds. That could just be a WASI level proposal. Should we define a new proposal template for a world.
Luke Wagner: From my perspective, makes sense to have repo as world. Possibly have worlds and itnerfaces separated by worlds. Other case, couple HTTP interface with HTTP proxy world. Don’t want to always separate them, may have collection of interfaces and worlds in the same repo.
Dan Gohman: Makes sense, I’ll have a new proposal for our next meeting
Dan Gohman: Just calling attention to the PR.
Dan Gohman: POSIX historically included. A lot of languages have filtered those out because users don’t care about those. Idea is that we could also filter these out and then re-add them. Then there’s a question of what Preview 1 should do. Could just continue to do the POSIX conforming. Curious about sg
Pat Hickey: Because there are existing P1 programs, we shouldn’t make changes like this.
Dan Gohman: I’m seeing sam as a thumbsup up there. More broadly, I think any changes to P1 should increase compat and we should try not to introduce non-determinism. That sound right? Looks like agreement