Update: yes, lixcb-randr0 is installed and RandR=1280×800 is now set in the options of my RealVNC Server 7.10.0.52269 for arm64.

But no change, same error.

Just to add: 13.3.1 ran flawlessly w/o setting RandR or anything else.

This works fine for me, using the built-in realvnc server paclage

vncserver-virtual -RandR=1280x800
RealVNC(R) Server 7.5.1 (r50075) ARMv8-A (May 30 2023 13:27:57)
Copyright (C) RealVNC Ltd.

OS: Debian GNU/Linux 12, Linux 6.6.20+rpt, aarch64

I think it is not so much the actual version of the package, but whether a virtual desktop is used, as opposed to the existing X session.

The following seems to work

mkdir -p ~/.config/systemd/user

echo \
"[Unit]
Description=RealVNC server in virtual mode
After=network.target

[Service]
Type=forking
ExecStart=/usr/bin/vncserver-virtual -RandR 1280x800 :1
ExecStop=/usr/bin/vncserver-virtual -kill :1

[Install]
WantedBy=default.target" \
> ~/.config/systemd/user/realvnc-virtual.service

systemctl --user daemon-reload
systemctl --user enable --now realvnc-virtual

Then connect your VNC viewer to port 5901 and launch Mathematica.

Yes, works for me, too, thanks.

Now I'm looking to get it running not only via your idea with vncserver-virtual in ~/.config/systemd/user/realvnc-virtual.service but also via the standard vncserver-x11-serviced from /lib/systemd/system/vncserver-x11-serviced.service; for the latter, RandR is set but the error still comes up...

Best regards, Michael

P.S.: In the meantime, I've also upgraded to the latest version, 7.11, on both server and client side, but no change.

By the way, you don't necessarily need the standalone wolframscript package any more. There is an up to date version of wolframscript included in the wolfram-engine package

Setting up wolfram-engine (14.0.0+202403231274) ...
update-alternatives: using /opt/Wolfram/WolframEngine/14.0/SystemFiles/Kernel/Binaries/Linux-ARM64/wolframscript to provide /usr/bin/wolframscript (wolframscript) in auto mode

The latest version of the wolframscript package has been published in the official repository.

While it is not necessarily needed or useful when wolfram-engine is already installed, there is also the possibility of cloud kernel use, i.e. wolframscript -cloud without a full local installation.

I think Stefan meant the libxcb-randr0 package