Using VirtualGL with a vnc server

[wiak]

Using VirtualGL with a vnc server (tigervnc) to provide fast graphics for low graphics capability client computers

https://wiki.archlinux.org/title/VirtualGL
https://cdn.rawgit.com/VirtualGL/virtua ... html#hd006
https://wiki.archlinux.org/title/TigerVNC

I earlier decribed the use of VirtualGL back in Dec 2020, but unfortunately some of what I did back then was totally unnecessary (e.g. no need to use vglconnect, which actually sets up an X11 connection rather than vnc). That erroneous write-up can be found here (haven't re-written it yet since still contains some useful info): https://weedoglinux.rockedge.org/viewto ... p=263#p263

However, recently one of my family needed to almost permanently borrow the HP Elitebook Folio G1 1020 G1 with Core m vPro processor and 8GB RAM laptop I do most of my WeeDogLinux development work on. So I am back using old HP Elitebook 2530p (core2duo 4GB RAM machine from 2008...). Problem is that the graphics card in that old 2530p machine is only OpenGL 2.1 capable whereas, latest Blender, for example, requires a minimum of OpenGL 3.3 to run. i.e. It is impossible to run latest Blender (and many other graphics intensive apps) on the 2530p at all. Solution is to run Blender on a server that has an OpenGL 3.3 or above capable graphics card and simply use the 2530p as a viewer.

First thought might be that you could use the Xserver of the 2530 via say ssh -X (X forwarding), but unfortunately that doesn't work because the OpenGL 3.3+ processing instructions actually then get sent (along with X11 protocol info) back to the 2530p computer, so same issue - it cannot display the result...

Part of the actual solution is to use vnc (tigervncserver on the Folio G1 and tiger vncviewer on the 2530p). However, if you simply use vnc in that manner you will find that the frame rate is hopelessly slow for using the likes of Blender (or say, Supertuxkart racer game...). Why? Well unfortunately software like Blender when run from main login X display (usually DISPLAY :0) uses the graphics hardware directly. Alas, the usual Xvnc virtual X server does not use the 3D accelerated graphics available directly (so normal Xvnc rendering is 2D software slow rendering). VirtualGL however intercepts the OpenGL calls and redirects them to the graphics adapter for processing there prior to returning the rendered result back to the Xvnc vncserver and the resultant speed up is enormous... such that fast OpenGL games can even be played and controlled via vncviewer on the slow computer thin client. Arch Linux shows diagrammatically how VirtualGL works:

Code: Select all

   server:                                                              client:
 ······················································               ················
 : ┌───────────┐ X11 commands         ┌─────────────┐ : image stream  : ┌──────────┐ :
 : │application│━━━━━━━━━━━━━━━━━━━━━▶│VNC server 2)│━━━━━━━━━━━━━━━━━━▶│VNC viewer│ :
 : │           │        ┌───────────┐ └─────────────┘ :               : └──────────┘ :
 : │           │        │X server 1)│        ▲        :               :              :
 : │ ╭┈┈┈┈┈┈┈┈┈┤ OpenGL │ ╭┈┈┈┈┈┈┈┈┈┤ images ┃        :               :              :
 : │ ┊VirtualGL│━━━━━━━▶│ ┊VirtualGL│━━━━━━━━┛        :               :              :
 : └─┴─────────┘        └─┴─────────┘                 :               :              :
 ······················································               ················

After I first published about using VirtualGL, forum member rufwoof pointed out an alternative - using x0vncserver instead of the usual tiger Xvnc vncserver. Like I said, the main desktop X (usually using DISPLAY :0) does involve direct 3D rendering. As it happens tigervnc provides alternative vncserver called x0vncserver which simply sends back the already rendered DISPLAY :0 X display to the client vncviewer, which gives that solution probably similar performance to using normal Xvnc vncserver with VirtualGL. I tried the command rufwoof suggested to use x0vncserver, and it did seem to work similarly well albeit only from that one actual X DISPLAY:0 screen. Here is the command:

Code: Select all

DISPLAY=:0 x0vncserver -rfbauth ~/.vnc/passwd

Note that I didn't have appropriate Xauthority permissions with this method, so on the server computer used the risky “xhost +” method to allow access to the X display from any client...

However, in practice, my whole family wants individual access to the same server, so x0vncserver doesn't provide that really (well, I could run true X several times on different DISPLAYs I suppose, but seemed messy, if possible). VirtualGL, however, works well with Xvnc so I decided to use that (and run several vncserver instances, one for each client).

Originally, I used VirtualGL on BionicDog64 with no login Display Manager and more recently repeated the exercise using WDL_Arch64, which by default didn't have any login DM either. However, then I also needed to use "xhost +" on the server machine to allow the clients to use the server vnc X displays. The better way, however, is to use an actual login DM. Hence I installed lightdm per step 1 below. Great thing about that is that VirtualGL configuration program automatically configures lightdm to provide the required server X authorities to the vncviewer client programs. Note that SLiM display manager wouldn't be sufficient - it doesn't work with remote login connections. Note also, however, that all the main configuration work is done on the server machine. All the client machines simply need tiger vncviewer installed, so easy... (yes, could even use an Android phone with suitable vncviewer, though not sure would work as good as tiger vncviewer or turbo vncviewer...)

Anyway, to make end of this long story a bit shorter, here are the steps I used to create the overall vnc+VirtualGL driven system:

1. Install tigervnc, virtualGL, and lightdm on the Folio G1 server machien (in my case using WDL_Arch64 distro):

Code: Select all

pacman -Sy tigervnc virtualGL lightdm lightdm-gtk-greeter

NOTE that if you don't use lightdm you would have to run command: xhost + on the server (Folio G1) for all of this to work. But using lightdm provides automatic X authentication to all clients involved so very convenient indeed.

2. Run vglserver_config command (as root) in a terminal:
choosing Yes for each of the three questions:

Code: Select all

Restrict 3D X server access to vglusers group (recommended)?
[Y/n]y
Restrict framebuffer device access to vglusers group (recommended)?
[Y/n]y
Disable XTEST extension (recommended)?
[Y/n]y

VirtualGL automatically then creates the following:

Code: Select all

... Creating vglusers group ...
groupadd: group 'vglusers' already exists
Could not add vglusers group (probably because it already exists.)
... Creating /etc/opt/VirtualGL/ ...
... Granting read permission to /etc/opt/VirtualGL/ for vglusers group ...
... Creating /etc/modprobe.d/virtualgl.conf to set requested permissions for
    /dev/nvidia* ...
... Granting write permission to /dev/dri/card0 for vglusers group ...
... Modifying /etc/X11/xorg.conf.d/99-virtualgl-dri to enable DRI permissions
    for vglusers group ...
... Adding greeter-setup-script=vglgenkey to /etc/lightdm/lightdm.conf ...

Done. You must restart the display manager for the changes to take effect.

I use systemd to auto-start lightdm on boot so I just needed to reboot... but could have simply entered command:
systemctl restart lightdm
More generally, after most any of major part of the configuration, I just rebooted to get everything activated, but quicker, with systemd at any rate is just to employ that systemctl command appropriately (don't you wish you were using systemd?... it is very convenient...).

3. Add users onto the server machine that want access via vncviewer:

Code: Select all

useradd -m -G wheel,vglusers -s /bin/bash jack
useradd -m -G wheel,vglusers -s /bin/bash jill

... etc etc ... for as many users as you want to accommodate
I also put them in wheel group which I have set up not to need passwd with sudo.

4. Configure tigervnc server component. The somewhat involved procedure is well documented for Arch Linux at:
https://wiki.archlinux.org/title/TigerVNC#Installation
Basically you do this:
a. Create a password using vncpasswd which will store the hashed password in ~/.vnc/passwd. You need to do this for each user after logging in one by one (i.e. for jack and jill etc etc).
b. Edit /etc/tigervnc/vncserver.users to say which user will be allocated which X DISPLAY on the server (Folio G1) computer. My /etc/tigervnc/vncserver.users file simply contains:
:1=jack
:2=jill
Well it would do, if the users were jack and jill...
c. For each user (e.g. jack and jill) create file ~/.vnc/config to say what desktop (session type) they want and whatever. Since WDL_Arch64 comes with openbox for all users that ~/.vnc/config file simply contains the two lines:
session=openbox
alwaysshared

5. I use systemd to autostart my vncservers. So for jack and jill, I have entered commands:
systemctl enable vncserver@:1 # for jack, and
systemctl enable vncserver@:2 # for jill

6. You also need to install tigervnc on the client (slow 2530p machine in my case), but only because you need tiger vncviewer. And alternative is turbo vncviewer, but other vncviewers are not optimised for use with VirtualGL as far as I am aware so I recomment tiger...

7. Optionally (I think) you might want to employ a ssh tunnel. That requires you to edit /etc/ssh/sshd.config to include the lines:
AllowGroups vglusers
X11Forwarding yes

But you don't really need ssh for this vnc + VirtualGL purpose if you don't want that extra security I think.
If you do, on WDL_Arch64 I autostart sshd on reboot having used command: systemctl enable sshd
Isn't systemd simple and wonderful?

I think that is ‘almost’ about all there is to it (well, yes, it is pretty involved to set up first time and took me ages/days getting it all wrong many times prior to sorting out the above procedure).

Simply reboot. The VirtualGL bit needs no further action (even with systemd, no systemctl required for that bit - its all set up automatically when you ran command vglserver_config earlier).

8. To actually then use the system from your poor old client machine simple run:

Code: Select all

vncviewer

and if you are jack (who was allocated vnc DISPLAY:1) simply enter in the vncviewer window:

Code: Select all

ip_address_of_server_machine:1

and, all going well, a pop-up window will appear asking you for the vncserver passwd you entered (when as jack you used the vncpasswd command). Enter whatever that was and the window containing the full openbox desktop of user jack on the server should appear.

9. But the main trick now is, to run apps such as Blender or SuperTuxKart, do the following in that vnc desktop window:
Open a terminal and enter command (for example):

Code: Select all

vglrun blender

# or... vglrun supertuxkart (assuming that is installed on the server of course).

That vglrun command tells VirtualGL to intervene and arrange for the direct rendering to be done and then sent on to the vncserver virtual X display and then back to the vncviewer. Like magic, suddenly the old slow computer has the likes of Supertuxkart running smoothly at fast fram rate!!! By the way I even found this works fine (with both blender and supertuxkart) when using an ancient Pentium mobile 750MB RAM machine as the client via tiger vncviewer!!! Both apps also appeared fast and smooth enough on old Atom netbook using this technique. Truly amazing.

Note that you can run the server computer 'headless (i.e. without monitor)' if you wanted to. It's actually not recommended you also directly use the server, but I do and fine so far (albeit usually logged in as a different GUI desktop user, which is mainly root user at the moment, though I'm meaning to end that old habit...).

Animated GIF created via wex screen recorder which included button to convert screencast video using fredx181's gifenc-sel program. The actual program runs very smoothly with good frame rate - choppiness in the gif is just cos it's a gif...

[Grey]

However, recently one of my family needed to almost permanently borrow the HP Elitebook Folio G1 1020 G1 with Core m vPro processor and 8GB RAM laptop I do most of my WeeDogLinux development work on.

I suddenly remembered how in the early 2000s I worked for a small TV company. There were not enough computers. We had to connect four keyboards, four mice and four monitors to one system unit via a hub. And the program (under Win, I forgot which one, paid and Russian) shared it all for four employees

[Eastler_Dart]

. . . for newest blender on grafic-hardware, which doesn't do OpenGL => 3

simply start blender (2.93) with a script, its content should be:

LIBGL_ALWAYS_SOFTWARE=true ./blender

( = put in the environment-Variable
LIBGL_ALWAYS_SOFTWARE
the value: true
and start blender
It starts in all cases
with software-GL, didn't search for
hardware-driver GL >= 3.0,
didn't break start because of error
'no compatibe Hardware driver found'
)

I had successful compiled blender 2.93
under an extended Upup-EF, in 32 bit,
it starts, im working there on a new blender-theme

hope it helps a little - Eastler

[user1111]

The method as described by @wiak (VirtualGL) works incredibly well. Combined with sndiod compiled/installed on the client system and replacing starting of x0vncserver on the host/server with straight vncserver, using vncviewer on the client to login to a terminal screen (or whatever you have defined in the servers ~/.vnc/xstartup) and then

xhost +
vglrun jwm

and youtubes etc play as good as if played locally. Can even watch films full screen with high screen actions in good visual quality.

Running for instance
AUDIODEVICE=snd@<ip of client>/0 vlc

and with sndiod running on the client
sndiod -dddd -a on -f rsnd/1 -L-
(for card 1, change to rsnd/0 for card 0) and you see whatever vlc plays whilst sound also comes through beautifully.

Considerably superior to x0vncserver IMO. Better still if you export that AUDIODEVICE .... globally, so anything started in jwm equally forwards sound. And yes, one sndiod feed doesn't block others, for instance with a youtube playing running
AUDIODEVICE=snd@192.168.1.7/0 aucat -i /dev/urandom
... results in the youtube still being heard but also a load of random noise also being blasted out.

In my case actually using a remote browser is quicker/better than if I run one locally on my laptop, as the laptop is wifi connected whilst the server is hard wired and has faster internet (and more cpu cores). Whilst the visuals/sounds are just as good. On a lower spec PC you'd be running with the equivalent of higher spec GPU type visuals.

Disagree with the systemD however, starting/restarting/stopping services can be just as easy within non-sysD. Then again I'm running OpenBSD at present, just the base system + tigervnc installed (connected to my Fatdog server from where I'm running jwm/chrome as above). One noticeable difference is that in OpenBSD when I mouse to screen edges the screen pans a lot smoother than it does when using Linux (sndiod sound also seems more crisp/nicer). Browser scrolling is also very smooth when using VGL, much smoother than if using x0vncserver.

[wiak]

Considerably superior to x0vncserver IMO. Better still if you export that AUDIODEVICE .... globally, so anything started in jwm equally forwards sound. And yes, one sndiod feed doesn't block others, for instance with a youtube playing running
AUDIODEVICE=snd@192.168.1.7/0 aucat -i /dev/urandom
... results in the youtube still being heard but also a load of random noise also being blasted out.

In my case actually using a remote browser is quicker/better than if I run one locally on my laptop, as the laptop is wifi connected whilst the server is hard wired and has faster internet (and more cpu cores). Whilst the visuals/sounds are just as good. On a lower spec PC you'd be running with the equivalent of higher spec GPU type visuals.

I was worried VirtualGL wouldn't keep being actively developed, but seems it is. I haven't myself tested using sound delivery mechanisms along with it, but must try that sometimes.
My family main need for vgl was that we had low powered client machines with poor OpenGL support so was huge benefit to run apps on the server but with VirtualGL to use the server Graphics Card, not just standard vnc where opengl stuff left for the local machine (as far as I remember it). We are all now currently using more powerful laptops but I continue to like the idea of being able to run via thinnish clients to a server that, as you say, may also be hard-wired to Internet router rather than just via wifi.

[Clarity]

@wiak VNC, by the accounts of its authors never is intended to be a Remote Desktop, rather it IS INTENDED to be a remote display; thus sound (or aka multimedia) is not in its design criterion.

XRDP has been in Ubuntu for a decade with active support. Its design is to use the compressed-encrypted protocol in a Linux environment to serve same as is OOTB in Wins as well as the MACs which fire it up to allow desktop connections. Today, you can get a full desktop as if you were sitting at the Remote's console listening to whatever it might be playing, watching whatever you wish, and interacting with the remote desktop any way of choice. Tested this in Ubuntu in the past couple years ago. Its worth a look, IMHO and should not require any developer code for its operations.

Clients for connection are in WoofCE PUPs as well in most Unix/Linux OS so no client-side installations needed.

As we move to Pipewire, I am curious if there are issues. Will try to investigate in the coming days by firing up the latest Ubuntu to test desktops access again.

Videos to see it in operation are across the internet.

This is merely FYI

[wiak]

@wiak VNC, by the accounts of its authors never is intended to be a Remote Desktop, rather it IS INTENDED to be a remote display; thus sound (or aka multimedia) is not in its design criterion.

XRDP has been in Ubuntu for a decade with active support. Its design is to use the compressed-encrypted protocol in a Linux environment to serve same as is OOTB in Wins as well as the MACs which fire it up to allow desktop connections. Today, you can get a full desktop as if you were sitting at the Remote's console listening to whatever it might be playing, watching whatever you wish, and interacting with the remote desktop any way of choice. Tested this in Ubuntu in the past couple years ago. Its worth a look, IMHO and should not require any developer code for its operations.

.

RDP is just a protocol. The information needed is what app are you using for remote desktop in Ubuntu (which may well use XRDP)?
VNC is one method to get remote display, for sure, but that can be used as part of remote desktop control. Open source versions of VNC don't provide audio, but may provide file transferring. But you can always add in the use of secure shell client/server with sftp or rsync for file copying/sharing, and pulseaudio or pipewire for audio (but if these lag once of the sndio or trc audio methods others have discussed on this forum - I haven't checked these app/service names so apologies in advance - just typing from memory). Certainly work is involved getting all such pieces to work together, which is indeed a bit unsatisfactory. So question is: what app is Ubuntu using that your refer to. No reason I can imagine that same app couldn't be used in distros here.

So maybe I answer my own question via this link I found: https://itsfoss.com/xrdp-ubuntu/

Does it handle audio too?

[Clarity]

Does this help? XRDP is in the repo for Linuxes. This is not new technology and the protocol for secure full-function desktop transfer has been around for almost 25 years. Windows, Intel and IBM started it, then MACs signed on, then Linux via XRDP serving its multimedia desktops via the same efficient protocol.

No desktop apps are involved/modified/changed at the XRDP server end. Clients connect to experience/use the full desktop offering from the server.

Again, this is information to give an understanding of the difference between what one gets from an RDP server vs the limited one gets from a VNC server. Their designs differ because of the advantage the protocol provides vs, say, X11 over the LAN.

Hope this info helps to see the difference and benefit.

[wiak]

Does this help? XRDP is in the repo for Linuxes. This is not new technology and the protocol for secure full-function desktop transfer has been around for almost 25 years. Windows, Intel and IBM started it, then MACs signed on, then Linux via XRDP serving its multimedia desktops via the same efficient protocol.

No desktop apps are involved/modified/changed at the XRDP server end. Clients connect to experience/use the full desktop offering from the server.

Yes and no, I'd say. That script the guy uses in that youtube example (ignore that was for arm devices) handles audio via:

# xRDP4arm
...
This install.sh script will install xrdp, lxde, pulseaudio and create a remote rdp user for your Ubuntu 20.04 machine.

i.e. it simply uses pulseaudio for the audio part of the connection. Some here have suggested that there is a significant audio lag (in relation to the video) involved using pulseaudio for this (and I have tried pulseaudio for that too and found similar result). That is why some of the alternative audio fetch techniques discussed on this forum (sndiod and trx) 'might' provide a better remote desktop solution as far as the audio part goes. VNC actually works very well indeed for the video/desktop component (especially when paired with virtualGL - very fast...).

https://sndio.org/

The sndiod daemon is an intermediate layer between audio or MIDI programs and the hardware. It performs the necessary audio processing to allow any program to work on any supported hardware. By default, sndiod accepts connections from programs running on the same system only; it initializes only when programs are using its services, allowing sndiod to consume a negligible amount of system resources the rest of the time. Systems with no audio hardware can use sndiod to keep hot-pluggable devices usable by default at virtually no cost.

But can access sndiod via network:

-L addr
Specify a local network address sndiod should listen on; sndiod will listen on TCP port 11025+n, where n is the unit number specified with -U. Without this option, sndiod listens on the UNIX-domain socket only, and is not reachable from any network. If the option argument is ‘-’ then sndiod will accept connections from any address. As the communication is not secure, this option is only suitable for local networks where all hosts and users are trusted.

http://www.pogo.org.uk/~mark/trx/
https://github.com/eugenehp/trx/wiki/tr ... io-over-IP

trx is based on the following software/libraries:

ALSA (Linux)
Opus codec
lib oRTP: packaged with most Linux distributions

There is also trix, which is based on trx:
https://github.com/JorenSix/trix

trix is a simple toolset for broadcasting live audio from Linux or macOS. It sends and receives encoded audio over IP networks, via an audio interface. If audio interfaces are properly configured, a low-latency point-to-point or multicast broadband audio connection can be achieved. This could be used for networked music performances.

Audio Transmitter/Receiver over Ip eXchange is based on trx by Mark Hills . The main difference being that it supports ALSA (Linux), PulseAudio (Linux) and CoreAudio (macOS) thanks to the inclusion of the intermediate rtAudio library.

It appears from Arch Wiki that it may be possible to use pulseaudio without typical TCP stream lags:
https://wiki.archlinux.org/title/PulseA ... er_network

Using RTP/UDP instead of native-protocol-tcp
There are serious issues with trying to send data in real time over TCP, especially over lossy connections like wifi. This is why RTP over UDP was invented. It can be used to increase reliability and reduce latency.

When RTP is working properly, late or dropped packets will just create a few milliseconds of silence instead of a long pause while TCP is orchestrating the packet resend logistics. As an added bonus, if the remote server is ever restarted, the connection will be re-established automatically. However there will no longer be a way to remotely control the server's master volume, with each client machine having its own independent master volume instead.

To use RTP instead of native-protocol-tcp, pulseaudio clients must connect to a local pulseaudio server first. This local server then connects to the remote pulseaudio server through RTP.

I haven't checked at all, but I wouldn't be surprised if some larger Remote control applicaions use pulseaudio with RTP/UDP in above manner.