PVF host: Specialize on Linux, support macOS

[pepyakin]

(This is not an actionable issue, just a platform for discussion and linking from other issues)

To my knowledge, we do not support Windows explicitly. Maybe mingw may work, but MSVC almost surely won't work. That's fine since Windows has WSL for those who need to run Polkadot there.

There is quite a bunch of people who are using macOS for development either of Polkadot or parachains. While running Polkadot is possible inside a docker container, it comes with many inconveniences:

• polkadot-launch does not support docker,
• macOS does not support docker natively so it spins up a virtual machine, which can eat a decent amount of memory and CPU, and then on top of that the host and the guest do not share the resources.
• profiling inside of a docker is not the best experience on macOS either

So therefore we probably want to keep supporting polkadot directly on macOS.

However, PVF host or other parts of wasm execution may want to use special features of Linux for performance, security or other reasons. To name a few:

1. Linux supports uffd which is useful for low-latency instantiation in wasmtime. Maybe macOS' mach layer supports for something like that, but it would be take additional effort to implement that.
2. we are interested in security hardening of the worker processes PVF host: sandbox/harden worker process #882. Even if macOS supports something of that sort, it would take us to reimplement the same for it.
3. we may need to constrain the workers with amount of resources they can take. The best we can hope on macOS is probably rlimit (and the last time I looked I wanted to use cgroups, Linux only feature). And again if there are required facilities, we would have to reimplement everything for macOS again.

Thus, it may make sense to specialize our implementation efforts on Linux for production deployments. We can implement good enough fallbacks that would allow deploying simple localnets, but running validators on macOS will be discouraged.

[mrcnski]

Due to #882 becoming high-priority for parathreads, we are now forced to specialize on Linux. But we will still support MacOS with e.g. an --insecure-validator-mode flag.

[mrcnski]

Now that we use Linux landlock (paritytech/polkadot#7303) we've taken the first step down this path.

[nazar-pc]

To my knowledge, we do not support Windows explicitly. Maybe mingw may work, but MSVC almost surely won't work. That's fine since Windows has WSL for those who need to run Polkadot there.

For us at Subspace Windows support is essential for user adoption (tens to hundreds of thousands of consensus nodes), we use Substrate as a library, including in GUI desktop apps and run software on Windows in addition to macOS and Linux. It compiles with MSVC and works just fine (modulo rare minor issues from time to time that we report back and that are getting fixed fairly quickly).

So while it is not really officially supported (that was mentioned multiple times), it is important for us that it remains available and working, we also run it in CI all the time and BTW would be great if polkadot-sdk also had Windows in CI just to make sure things at least compile.

[mrcnski]

@nazar-pc Thanks for raising the concern. This issue is just about validator nodes, which are (and have been for a long time) Unix-only. They already are not expected to compile on Windows.

would be great if polkadot-sdk also had Windows in CI just to make sure things at least compile.

If substrate is expected to support Windows then that could be worth doing just for substrate. Can you please raise a separate issue, if there isn't one?

[bkchr]

If substrate is expected to support Windows then that could be worth doing just for substrate. Can you please raise a separate issue, if there isn't one?

Will not be done. We are open to merge fixes for Windows, but we will not have any CI job.