virtio-gpu device

[dorindabassey]

Hi,

I and @MatiasVara are interested in developing a virtio-gpu device for rust-vmm.
We have just started working on an initial skeleton. As soon as we have something ready, we'll open a Draft PR.
The idea is to have a rust based virtio-gpu device working. Marc-Andre Lureau will be assisting us in this effort, given that he was involved in the development of the vhost-user-gpu C implementation in QEMU.

[stsquad]

Hi @dorindabassey et all,

We are certainly interested in a virtio-gpu backend for vhost-device. We are currently working towards a demo at our next Connect to bring together Arm, Xen and virtio-gpu (venus/vulkan) on the AVA platform. We are working with QEMU in the first instance but longer term having a separate rendering process makes sense, especially on Xen. What are your thoughts about supporting virgl and venus vs other gfxstream transports like wayland?

[dorindabassey]

@stsquad currently we are starting looking at virgl first to try to match the existing vhost-user-gpu in qemu, we would like to support all those other features but we will see later how the project evolves.

Are there somethings you think we should consider while working on this?

[MatiasVara]

I add @elmarco to the conversation.

[DemiMarie]

@dorindabassey How will this compare to the existing virtio-GPU device implemented in crosvm? crosvm already has vhost-user support. Blob objects (needed for Vulkan and native contexts) require non-standard vhost-user messages, but I don’t think that an alternative implementation would be able to avoid this.

Also paging @alyssais who is using this in Spectrum OS.

[dorindabassey]

Hi @DemiMarie, the vhost-device-gpu implementation we are working on is largely based on crosvm virtio-gpu device, given that we are using rutabaga+virglrenderer. We are not doing anything special other than how the display is being handled, we are going to be use QEMU display support for this. I hope that answers your question.

[dorindabassey]

update: adding @mtjhrc who is also working on the vhost-device-gpu with us.

[alyssais]

We are not doing anything special other than how the display is being handled, we are going to be use QEMU display support for this.

This is surprising to me, compared to using crosvm's display code in the backend. If this is a vhost-user backend, how does the device interact with the QEMU display code? What happens if the vhost-user frontend is not QEMU?

[DemiMarie]

@alyssais, @QubesOS, and Chrome OS all need each window in the guest to be a separate window on the host. Therefore, they are or will be using Wayland passthrough, rather than QEMU’s display code.

[mtjhrc]

We are not doing anything special other than how the display is being handled, we are going to be use QEMU display support for this.

This is surprising to me, compared to using crosvm's display code in the backend. If this is a vhost-user backend, how does the device interact with the QEMU display code? What happens if the vhost-user frontend is not QEMU?

It interacts using commands on socket obtained from VHOST_USER_GPU_SET_SOCKET , it is documented here: https://www.qemu.org/docs/master/interop/vhost-user-gpu.html

If the fronted doesn't support VHOST_USER_GPU_SET_SOCKET, then it wouldn't work, but we could also support Wayland passthrough too, I guess.

[alyssais]

It interacts using commands on socket obtained from VHOST_USER_GPU_SET_SOCKET , it is documented here: https://www.qemu.org/docs/master/interop/vhost-user-gpu.html

Ah, I see, so this is an implementation of QEMU's vhost-user-gpu protocol (which has the frontend do the rendering), not crosvm's (where the backend does it). It's very unfortunate that there's no consistency/standardisation between the two…

[DemiMarie]

Having the backend do the rendering is highly desirable from a security perspective, as it allows the frontend to be sandboxed. The backend must be quite privileged as it can directly access the host’s GPU drivers.