PVF host: Investigate how to better launch worker processes

[pepyakin]

Right now the PVF validation host spawns worker processes for doing compilation of wasm blobs and executing wasm code during validation of candidates.

The processes are spawned by taking the polkadot executable and invoking it via Command with additional parameters that specify the type of the worker and the path to the IPC socket that the worker will receive commands with. At the same time, the polkadot binary provides command-line arguments that would pass the control to the workers if a special worker-type command was passed.

This proved to be an awkward way of doing things. It is one of the issues what prevents us from testing the validation host thoroughly (#2342) and already shot us in the foot (e.g. this).

I did a preliminary research on how to do the things better.

First what comes to mind is fork(2). We fork the process and then pass the control to the worker entrypoint. However, this seems to be rather fragile. The reason is that fork preserves the memory content of the process, it's opened fds and at the same time only the thread that called fork will be running. The implication of that, is that the state of the program in the child preserves all the mutexes and the control flow can be anywhere. One of obvious examples of what can go wrong, is that if another thread called malloc and got the lock then if the thread that called the fork calls malloc as well, then it will get deadlocked. So therefore, only async-signal-safe are callable. Theoretically this may be solved by resuming threads but that's also obviously a bad idea: an example problem is that the same thread will write into the same descriptor in both parent and child process. That can corrupt data and send gibberish into sockets.

vfork is a special case of fork which is typically followed by execve. The latter basically the primitive that is used by Command and requires the filename to execute, which basically brings us to the same problem as the existing solution.

If we step back and think what we actually need here we would notice that we just need to reuse the same executable image but specify a different entrypoint. Ideally, this entrypoint should be represented by a function address and would not require the executable to handle special arguments for invoking the required worker.

On Linux, there is a clone(2) syscall which is basically a pumped up fork. It allows for specifying the entrypoint as a function pointer and it also takes flags to specify what parts of the parent to be carried on to the child process. The parent can either share or not share the address space, can put the child into new namespace, and so on. (In fact, using this syscall is how pthread on Linux is implemented).

If we were to use it it would also allow us to introduce sandboxing of the worker processes, a feature envisioned in the initial design of the overhauled PVF validation host.

So in our case the parent will invoke clone, sharing almost nothing with the child process, passing an entrypoint to the required worker.

However, there is a problem with that. Even though Rust has a minimal runtime, it still has to be initialized. For that, we must make sure that the pre-main stuff is executed as if we were invoking a program but with a different main entrypoint. I am not sure if clone(2) handles that by default or if that should be taken care of manually. In the end, C also has pre-main and from what I have seen in C there is no special fluffing needed. In any case, this should be investigated.

One implication of going this way, is that clone(2) is Linux only syscall with no obvious (to me) alternatives in macOS, and we still want to support that. There is this discussion about that #881