Add story of Alan running in trouble with large stack allocations #38
Conversation
| * Barbara, as an expert in Rust, may have had the tools to understand that `pin_mut` is used for pinning to the stack while `Box::pin` is for pinning heap allocations. | ||
| * This problem is somewhat subtle, so someone like Niklaus would probably have had a much harder time figuring this out (or even getting the code to compile in the first place). | ||
| * **Could Alan have used another API to achieve the same objectives?** | ||
| * Perhaps! Tokio's `select!` macro doesn't require explicit pinning of the futures it's provided, but it's unclear to this author whether it would have been smart enough to avoid pinning large futures to the stack. However, pinning is a part of the way one uses futures in Rust, so it's possible that such an issue would have arisen elsewhere. |
There was a problem hiding this comment.
It just pins things on the "async stack" automatically for you. The futures-rs version however does that too after I implemented rust-lang/futures-rs#1873 . The tokio version mainly goes a little bit further and avoids the FusedFuture requirements based off the ideas in rust-lang/futures-rs#1989.
I don't any version of the select! macro would resolve the issue on its own - one will always need an explicit Box to decouple the Futures. Only something like FuturesUnordered or other sub-executors will do it automatically.
Still surprised that the whole thing took 4Gb and could be fixed by improving one select. While it's known that generators can grow exponentially in size it seems to take a while to get to 4Gb 😯
@nikomatsakis This is the story I mentioned to you about large stack allocations caused by pinning large futures to the stack.