I wondered, Browsers work really well, are already there anyways, have all the GPU stuff etc already dealt with.

It could use the Browsers screencast ability on all platforms, and run with Javascript and WASM.

The stuff could be installed in a local Podman container and thus also work natively on Linux.

Do you know an app that does this, client-side?