I wrote a little OpenGL app in Rust and the hardest part was getting it to run on NixOS. This post describes the flake that works, and lists the errors that happen if the configuration isn’t right.
The full code for this post is available here (tag flake-blogpost
).
NixOS, per se, wasn’t the problem, but the complicated state of the graphics ecosystem, the runtime configuration that graphical apps do, and the weirdness of NixOS combined to make things not work out of the box.
This post is specifically about Rust apps built on NixOS 21.11, which use egui
, eframe
, glutin
, and OpenGL. That said, most of this should apply to building OpenGL apps on NixOS in general.
Table of contents
Flake overview
We’re going to write a NixOS flake that sets up a nix develop
shell environment for our Rust app, and also packages it so that it can be installed or nix run
.
Our flake begins with its inputs:
{
inputs = {
naersk.url = "github:nmattia/naersk/master";
nixpkgs.url = "github:NixOS/nixpkgs/nixos-21.11";
utils.url = "github:numtide/flake-utils";
flake-compat = {
url = github:edolstra/flake-compat;
flake = false;
};
};
From top to bottom, we import the following:
-
naersk
to build the Rust code, -
nixpkgs
for the Rust compiler and other libraries we need. We’re specifically using thenixos-21.11
branch, which matches the NixOS system version the flake must be run on. If the versions don’t match, we will probably get a runtime error about EGL contexts, -
flake-utils
for the functions that generalize our flake over multiple architectures, and -
flake-compat
so that our flake can be imported by non-flake nix. Specifically,flake.nix
lets users specify our repository in theinputs
of their flakes, anddefault.nix
lets users justimport ./.
the directory.
The full default.nix
is the following. Note that flake-compat
is actually unused in flake.nix
, but by listing it as a dependency, its version gets pinned in flake.lock
. We read this version in default.nix
, and run the code associated with it. This is better than hard-coding a version in the file, or using the latest version.
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/ .tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
)
{ src = ./.; }
).defaultNix
Next, we start specifying the outputs of the flake:
outputs = { self, nixpkgs, utils, naersk, ... }:
utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { inherit system; };
naersk-lib = pkgs.callPackage naersk { };
# ...
in
{
This is just the usual flake-utils
boilerplate to generalize the flake over multiple architectures. If we didn’t have this, we’d have to manually write our outputs for x86_64-linux
, aarch64-linux
, etc.
Finally, we get to the actual outputs of the flake:
defaultPackage = naersk-lib.buildPackage {
# ...
};
defaultApp = utils.lib.mkApp {
drv = self.defaultPackage." ";
};
devShell = with pkgs; mkShell {
# ...
};
});
}
The defaultPackage
is what users get if they install the flake. We use naersk
to build this. The defaultApp
is what gets called if users nix run
the flake. We use flake-utils
to run the defaultPackage
here. Lastly, devShell
is what users get if they run nix develop
, or have a direnv
with use flake
.
devShell
Let’s get the devShell
squared off because, without it, we can’t even build the app.
devShell = with pkgs; mkShell {
buildInputs = [
cargo
cargo-insta
pre-commit
rust-analyzer
rustPackages.clippy
rustc
rustfmt
tokei
xorg.libxcb
];
RUST_SRC_PATH = rustPlatform.rustLibSrc;
LD_LIBRARY_PATH = libPath;
GIT_EXTERNAL_DIFF = " /bin/difft";
};
devShell
part of flake.nix
We use mkShell
to specify the shell. The buildInputs
list all the packages that will be available inside the shell. We need the Rust build tools, and also libxcb
as the sole graphics system dependency of the app. I usually also include rust-analyzer
for editor support, clippy
for linting, tokei
for counting lines of code, and cargo-insta
for pretty cram tests.
We then set a few environment variables: RUST_SRC_PATH
is needed for some of the Rust tooling, and GIT_EXTERNAL_DIFF
configures git diff
to use difftastic
, which is nicer to read that vanilla diff
.
This is enough to cargo build
the app (after running nix develop
to enter the shell, or using direnv
to enter it automatically). But if we try to run the app, we’ll get an error. This is because most of the graphics libraries are actually loaded at runtime. This is also why the buildInputs
is surprisingly short, and doesn’t include any of the obvious suspects like libGL
.
Normally on Linux, libraries are installed in a few well known places, so the apps can find them without any additional configuration. However, we’re running NixOS, so we don’t allow any implicit dependencies. We have to specify the paths to our runtime deps explicitly:
libPath = with pkgs; lib.makeLibraryPath [
libGL
libxkbcommon
wayland
xorg.libX11
xorg.libXcursor
xorg.libXi
xorg.libXrandr
];
# ...
LD_LIBRARY_PATH = libPath;
flake.nix
We use makeLibraryPath
to turn our list of dependencies into a colon-separated string. The deps include both Wayland and X11 libraries because we don’t know what our users will be running. The app will discover this at runtime. Also, some of the X11 libraries are used in Wayland as well.
We put the string of deps into LD_LIBRARY_PATH
, which is where the dynamic linker will look for the libraries. With this, we should be able to cargo run
the app.
defaultPackage
At this point, our app is buildable and runnable. Let’s teach nix
how to do it itself by writing defaultPackage
:
defaultPackage = naersk-lib.buildPackage {
src = ./.;
doCheck = true;
pname = "sixty-two";
nativeBuildInputs = [ pkgs.makeWrapper ];
buildInputs = with pkgs; [
xorg.libxcb
];
postInstall = ''
wrapProgram "$out/bin/sixty-two" --prefix LD_LIBRARY_PATH : " "
'';
};
defaultPackage
part of flake.nix
We call naersk-lib.buildPackage
to build the app. Naersk reads the Cargo.toml
and Cargo.lock
files, and runs the Rust build commands as a Nix derivation. The src
, doCheck
, and pname
options are self-explanatory (docs). These would be sufficient for most Rust apps, but we’re doing graphics, so we need more configuration.
We have to mirror what we did in devShell
. First, we list libxcb
in buildInputs
so that it’s available to the Rust crates that need it. We don’t need to list the Rust build tools because Naersk does that itself.
We do however need to specify that our app needs several runtime libraries, and set LD_LIBRARY_PATH
. The basic way to do this is to generate a script like the following, which sets the environment variable, and “wraps” the real binary:
#!/bin/sh
export LD_LIBRARY_PATH="/path/to/lib1:/path/to/lib2:$LD_LIBRARY_PATH"
/path/to/real/exe "$@"
This is NixOS, so all the above paths would actually be nix store paths. We could write it by hand, but luckily there’s the wrapProgram
convenience function we can use instead. We import the function by adding makeWrapper
to nativeBuildInputs
, and then we call it in the postInstall
phase, which runs after cargo build
has finished.
That should do it: nix build
and nix run
should work on our repo.
Errors
Let’s now look at what sort of errors we get if we misconfigure something.
This section is here so that search engines will find this page when people search for the error strings. I hope it will save someone the time I spent trying to make sense of these obscure messages.
What happens if we don’t include libxcb
in buildInputs
?
error: linking with `cc` failed: exit status: 1
...
= note: /nix/store/8x25mhcdrnglaj55n80w8pnkwaqcp3sw-binutils-2.35.2/bin/ld: cannot find -lxcb
/nix/store/8x25mhcdrnglaj55n80w8pnkwaqcp3sw-binutils-2.35.2/bin/ld: cannot find -lxcb-render
/nix/store/8x25mhcdrnglaj55n80w8pnkwaqcp3sw-binutils-2.35.2/bin/ld: cannot find -lxcb-shape
/nix/store/8x25mhcdrnglaj55n80w8pnkwaqcp3sw-binutils-2.35.2/bin/ld: cannot find -lxcb-xfixes
collect2: error: ld returned 1 exit status
There are several xcb
libraries that we need. In NixOS, they’re all provided by the xorg.libxcb
package. On Fedora, it’s libxcb-devel
. On Debian, it’s several packages: libxcb1-dev
, libxcb-render0-dev
, libxcb-shape0-dev
, and libxcb-xfixes0-dev
What happens if we don’t include Wayland and X11 in LD_LIBRARY_PATH
?
thread 'main' panicked at 'Failed to initialize any backend! \
Wayland status: NoWaylandLib X11 status: \
LibraryOpenError(OpenError { kind: Library, detail: "\
opening library failed (libX11.so.6: cannot open shared object file: No such file or directory); \
opening library failed (libX11.so: cannot open shared object file: No such file or directory)" })', \
/home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/mod.rs:619:9
In NixOS, we have to include the libraries in LD_LIBRARY_PATH
. On Debian and Fedora, the app finds the libraries automatically.
Depending on which graphics system we’re running, we only need one set of libraries, but we have to include both for our users. This error also happens if $DISPLAY
or $WAYLAND_DISPLAY
are not set correctly, but that would be a window manager snafu.
What happens if we don’t include libxkbcommon
in LD_LIBRARY_PATH
?
thread 'main' panicked at 'internal error: entered unreachable code', /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/wayland/window/mod.rs:229:77
stack backtrace:
...
3: winit::platform_impl::platform::wayland::window::Window::new::...
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/wayland/window/mod.rs:229:77
...
12: wayland_client::imp::proxy::proxy_dispatcher
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/native_lib/proxy.rs:387:15
13: wl_closure_dispatch
14: dispatch_event.isra.0
15: wl_display_dispatch_queue_pending
16: wl_display_roundtrip_queue
17: wayland_client::imp::event_queue::EventQueueInner::sync_roundtrip::...
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/native_lib/event_queue.rs:85:17
18: scoped_tls::ScopedKey<T>::set
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137:9
19: wayland_client::imp::event_queue::with_dispatch_meta
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/native_lib/event_queue.rs:24:5
20: wayland_client::imp::event_queue::EventQueueInner::sync_roundtrip
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/native_lib/event_queue.rs:83:9
21: wayland_client::event_queue::EventQueue::sync_roundtrip
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/event_queue.rs:203:9
22: winit::platform_impl::platform::wayland::window::Window::new
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/wayland/window/mod.rs:229:21
23: winit::platform_impl::platform::Window::new
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/mod.rs:277:17
24: winit::window::WindowBuilder::build
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/window.rs:374:9
25: glutin::platform_impl::platform_impl::wayland::Context::new
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/wayland.rs:91:19
26: glutin::platform_impl::platform_impl::Context::new_windowed
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/mod.rs:113:20
27: glutin::windowed::<impl glutin::ContextBuilder<T>>::build_windowed
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/windowed.rs:341:9
28: egui_glow::epi_backend::create_display
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/egui_glow-0.17.0/src/epi_backend.rs:23:9
29: egui_glow::epi_backend::run
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/egui_glow-0.17.0/src/epi_backend.rs:56:27
30: eframe::run_native
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/eframe-0.17.0/src/lib.rs:180:5
31: sixty_two::main
at ./src/main.rs:21:5
32: core::ops::function::FnOnce::call_once
at /build/rustc-1.56.1-src/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[wayland-client error] A handler for wl_keyboard panicked.
We get the very classy internal error: entered unreachable code
.
It turns out that libxcbcommon
is needed at runtime even on Wayland. In NixOS, we have to include it ourselves. In Debian and Fedora, it just works.
What happens if we don’t include libGL
in LD_LIBRARY_PATH
?
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/api/egl/mod.rs:424:32
stack backtrace:
...
3: core::option::Option<T>::unwrap
at /build/rustc-1.56.1-src/library/core/src/option.rs:735:21
4: glutin::api::egl::Context::new
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/api/egl/mod.rs:424:19
5: glutin::platform_impl::platform_impl::wayland::Context::new_raw_context
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/wayland.rs:124:13
6: glutin::platform_impl::platform_impl::wayland::Context::new
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/wayland.rs:105:23
7: glutin::platform_impl::platform_impl::Context::new_windowed
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/mod.rs:113:20
8: glutin::windowed::<impl glutin::ContextBuilder<T>>::build_windowed
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/windowed.rs:341:9
9: egui_glow::epi_backend::create_display
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/egui_glow-0.17.0/src/epi_backend.rs:23:9
10: egui_glow::epi_backend::run
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/egui_glow-0.17.0/src/epi_backend.rs:56:27
11: eframe::run_native
at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/eframe-0.17.0/src/lib.rs:180:5
12: sixty_two::main
at ./src/main.rs:21:5
13: core::ops::function::FnOnce::call_once
at /build/rustc-1.56.1-src/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Again, in NixOS, we have to include this manually. In Debian and Fedora, the app finds them without extra work on our part.
We also get this Option::unwrap
panic if the system libraries are incompatible with what the app is using. This is why it’s important for the nixpkgs.url
to match the NixOS version the system is using.
Wrapping up
This post is long, but the flake itself is fairly short at only 66 lines, which includes several niceties like editor support and prettier diffing.
Like most NixOS things, this was hard to figure out, but once we know how to do it, it’s easy to adapt to different requirements.