I wrote a little OpenGL app in Rust and the hardest part was getting it to run on NixOS. This post describes the flake that works, and lists the errors that happen if the configuration isn’t right.

The full code for this post is available here (tag flake-blogpost).

NixOS, per se, wasn’t the problem, but the complicated state of the graphics ecosystem, the runtime configuration that graphical apps do, and the weirdness of NixOS combined to make things not work out of the box.

Thought bubble This post is specifically about Rust apps built on NixOS 21.11, which use egui, eframe, glutin, and OpenGL. That said, most of this should apply to building OpenGL apps on NixOS in general.

Flake overview

We’re going to write a NixOS flake that sets up a nix develop shell environment for our Rust app, and also packages it so that it can be installed or nix run.

Our flake begins with its inputs:

{
  inputs = {
    naersk.url = "github:nmattia/naersk/master";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-21.11";
    utils.url = "github:numtide/flake-utils";
    flake-compat = {
      url = github:edolstra/flake-compat;
      flake = false;
    };
  };

From top to bottom, we import the following:

  • naersk to build the Rust code,

  • nixpkgs for the Rust compiler and other libraries we need. We’re specifically using the nixos-21.11 branch, which matches the NixOS system version the flake must be run on. If the versions don’t match, we will probably get a runtime error about EGL contexts,

  • flake-utils for the functions that generalize our flake over multiple architectures, and

  • flake-compat so that our flake can be imported by non-flake nix. Specifically, flake.nix lets users specify our repository in the inputs of their flakes, and default.nix lets users just import ./. the directory.

The full default.nix is the following. Note that flake-compat is actually unused in flake.nix, but by listing it as a dependency, its version gets pinned in flake.lock. We read this version in default.nix, and run the code associated with it. This is better than hard-coding a version in the file, or using the latest version.

(import
  (
    let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
    fetchTarball {
      url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
      sha256 = lock.nodes.flake-compat.locked.narHash;
    }
  )
  { src = ./.; }
).defaultNix

Next, we start specifying the outputs of the flake:

  outputs = { self, nixpkgs, utils, naersk, ... }:
    utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
        naersk-lib = pkgs.callPackage naersk { };
        # ...
      in
      {

This is just the usual flake-utils boilerplate to generalize the flake over multiple architectures. If we didn’t have this, we’d have to manually write our outputs for x86_64-linux, aarch64-linux, etc.

Finally, we get to the actual outputs of the flake:

        defaultPackage = naersk-lib.buildPackage {
          # ...
        };

        defaultApp = utils.lib.mkApp {
          drv = self.defaultPackage."${system}";
        };

        devShell = with pkgs; mkShell {
          # ...
        };
      });
}

The defaultPackage is what users get if they install the flake. We use naersk to build this. The defaultApp is what gets called if users nix run the flake. We use flake-utils to run the defaultPackage here. Lastly, devShell is what users get if they run nix develop, or have a direnv with use flake.

devShell

Let’s get the devShell squared off because, without it, we can’t even build the app.

devShell = with pkgs; mkShell {
  buildInputs = [
    cargo
    cargo-insta
    pre-commit
    rust-analyzer
    rustPackages.clippy
    rustc
    rustfmt
    tokei

    xorg.libxcb
  ];
  RUST_SRC_PATH = rustPlatform.rustLibSrc;
  LD_LIBRARY_PATH = libPath;
  GIT_EXTERNAL_DIFF = "${difftastic}/bin/difft";
};
The devShell part of flake.nix

We use mkShell to specify the shell. The buildInputs list all the packages that will be available inside the shell. We need the Rust build tools, and also libxcb as the sole graphics system dependency of the app. I usually also include rust-analyzer for editor support, clippy for linting, tokei for counting lines of code, and cargo-insta for pretty cram tests.

We then set a few environment variables: RUST_SRC_PATH is needed for some of the Rust tooling, and GIT_EXTERNAL_DIFF configures git diff to use difftastic, which is nicer to read that vanilla diff.

This is enough to cargo build the app (after running nix develop to enter the shell, or using direnv to enter it automatically). But if we try to run the app, we’ll get an error. This is because most of the graphics libraries are actually loaded at runtime. This is also why the buildInputs is surprisingly short, and doesn’t include any of the obvious suspects like libGL.

Normally on Linux, libraries are installed in a few well known places, so the apps can find them without any additional configuration. However, we’re running NixOS, so we don’t allow any implicit dependencies. We have to specify the paths to our runtime deps explicitly:

libPath = with pkgs; lib.makeLibraryPath [
  libGL
  libxkbcommon
  wayland
  xorg.libX11
  xorg.libXcursor
  xorg.libXi
  xorg.libXrandr
];
# ...
LD_LIBRARY_PATH = libPath;
Parts of flake.nix

We use makeLibraryPath to turn our list of dependencies into a colon-separated string. The deps include both Wayland and X11 libraries because we don’t know what our users will be running. The app will discover this at runtime. Also, some of the X11 libraries are used in Wayland as well.

We put the string of deps into LD_LIBRARY_PATH, which is where the dynamic linker will look for the libraries. With this, we should be able to cargo run the app.

defaultPackage

At this point, our app is buildable and runnable. Let’s teach nix how to do it itself by writing defaultPackage:

defaultPackage = naersk-lib.buildPackage {
  src = ./.;
  doCheck = true;
  pname = "sixty-two";
  nativeBuildInputs = [ pkgs.makeWrapper ];
  buildInputs = with pkgs; [
    xorg.libxcb
  ];
  postInstall = ''
    wrapProgram "$out/bin/sixty-two" --prefix LD_LIBRARY_PATH : "${libPath}"
  '';
};
The defaultPackage part of flake.nix

We call naersk-lib.buildPackage to build the app. Naersk reads the Cargo.toml and Cargo.lock files, and runs the Rust build commands as a Nix derivation. The src, doCheck, and pname options are self-explanatory (docs). These would be sufficient for most Rust apps, but we’re doing graphics, so we need more configuration.

We have to mirror what we did in devShell. First, we list libxcb in buildInputs so that it’s available to the Rust crates that need it. We don’t need to list the Rust build tools because Naersk does that itself.

We do however need to specify that our app needs several runtime libraries, and set LD_LIBRARY_PATH. The basic way to do this is to generate a script like the following, which sets the environment variable, and “wraps” the real binary:

#!/bin/sh
export LD_LIBRARY_PATH="/path/to/lib1:/path/to/lib2:$LD_LIBRARY_PATH"
/path/to/real/exe  "$@"
Wrapper script

This is NixOS, so all the above paths would actually be nix store paths. We could write it by hand, but luckily there’s the wrapProgram convenience function we can use instead. We import the function by adding makeWrapper to nativeBuildInputs, and then we call it in the postInstall phase, which runs after cargo build has finished.

That should do it: nix build and nix run should work on our repo.

Errors

Let’s now look at what sort of errors we get if we misconfigure something.

Thought bubble This section is here so that search engines will find this page when people search for the error strings. I hope it will save someone the time I spent trying to make sense of these obscure messages.

What happens if we don’t include libxcb in buildInputs?

error: linking with `cc` failed: exit status: 1
...
  = note: /nix/store/8x25mhcdrnglaj55n80w8pnkwaqcp3sw-binutils-2.35.2/bin/ld: cannot find -lxcb
          /nix/store/8x25mhcdrnglaj55n80w8pnkwaqcp3sw-binutils-2.35.2/bin/ld: cannot find -lxcb-render
          /nix/store/8x25mhcdrnglaj55n80w8pnkwaqcp3sw-binutils-2.35.2/bin/ld: cannot find -lxcb-shape
          /nix/store/8x25mhcdrnglaj55n80w8pnkwaqcp3sw-binutils-2.35.2/bin/ld: cannot find -lxcb-xfixes
          collect2: error: ld returned 1 exit status

There are several xcb libraries that we need. In NixOS, they’re all provided by the xorg.libxcb package. On Fedora, it’s libxcb-devel. On Debian, it’s several packages: libxcb1-dev, libxcb-render0-dev, libxcb-shape0-dev, and libxcb-xfixes0-dev

What happens if we don’t include Wayland and X11 in LD_LIBRARY_PATH?

thread 'main' panicked at 'Failed to initialize any backend! \
Wayland status: NoWaylandLib X11 status: \
LibraryOpenError(OpenError { kind: Library, detail: "\
opening library failed (libX11.so.6: cannot open shared object file: No such file or directory); \
opening library failed (libX11.so: cannot open shared object file: No such file or directory)" })', \
/home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/mod.rs:619:9

In NixOS, we have to include the libraries in LD_LIBRARY_PATH. On Debian and Fedora, the app finds the libraries automatically.

Depending on which graphics system we’re running, we only need one set of libraries, but we have to include both for our users. This error also happens if $DISPLAY or $WAYLAND_DISPLAY are not set correctly, but that would be a window manager snafu.

What happens if we don’t include libxkbcommon in LD_LIBRARY_PATH?

thread 'main' panicked at 'internal error: entered unreachable code', /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/wayland/window/mod.rs:229:77
stack backtrace:
...
   3: winit::platform_impl::platform::wayland::window::Window::new::...
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/wayland/window/mod.rs:229:77
...   
  12: wayland_client::imp::proxy::proxy_dispatcher
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/native_lib/proxy.rs:387:15
  13: wl_closure_dispatch
  14: dispatch_event.isra.0
  15: wl_display_dispatch_queue_pending
  16: wl_display_roundtrip_queue
  17: wayland_client::imp::event_queue::EventQueueInner::sync_roundtrip::...
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/native_lib/event_queue.rs:85:17
  18: scoped_tls::ScopedKey<T>::set
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137:9
  19: wayland_client::imp::event_queue::with_dispatch_meta
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/native_lib/event_queue.rs:24:5
  20: wayland_client::imp::event_queue::EventQueueInner::sync_roundtrip
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/native_lib/event_queue.rs:83:9
  21: wayland_client::event_queue::EventQueue::sync_roundtrip
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-client-0.29.4/src/event_queue.rs:203:9
  22: winit::platform_impl::platform::wayland::window::Window::new
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/wayland/window/mod.rs:229:21
  23: winit::platform_impl::platform::Window::new
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/platform_impl/linux/mod.rs:277:17
  24: winit::window::WindowBuilder::build
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.26.1/src/window.rs:374:9
  25: glutin::platform_impl::platform_impl::wayland::Context::new
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/wayland.rs:91:19
  26: glutin::platform_impl::platform_impl::Context::new_windowed
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/mod.rs:113:20
  27: glutin::windowed::<impl glutin::ContextBuilder<T>>::build_windowed
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/windowed.rs:341:9
  28: egui_glow::epi_backend::create_display
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/egui_glow-0.17.0/src/epi_backend.rs:23:9
  29: egui_glow::epi_backend::run
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/egui_glow-0.17.0/src/epi_backend.rs:56:27
  30: eframe::run_native
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/eframe-0.17.0/src/lib.rs:180:5
  31: sixty_two::main
             at ./src/main.rs:21:5
  32: core::ops::function::FnOnce::call_once
             at /build/rustc-1.56.1-src/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[wayland-client error] A handler for wl_keyboard panicked.

We get the very classy internal error: entered unreachable code.

It turns out that libxcbcommon is needed at runtime even on Wayland. In NixOS, we have to include it ourselves. In Debian and Fedora, it just works.

What happens if we don’t include libGL in LD_LIBRARY_PATH?

thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/api/egl/mod.rs:424:32
stack backtrace:
...
   3: core::option::Option<T>::unwrap
             at /build/rustc-1.56.1-src/library/core/src/option.rs:735:21
   4: glutin::api::egl::Context::new
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/api/egl/mod.rs:424:19
   5: glutin::platform_impl::platform_impl::wayland::Context::new_raw_context
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/wayland.rs:124:13
   6: glutin::platform_impl::platform_impl::wayland::Context::new
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/wayland.rs:105:23
   7: glutin::platform_impl::platform_impl::Context::new_windowed
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/platform_impl/unix/mod.rs:113:20
   8: glutin::windowed::<impl glutin::ContextBuilder<T>>::build_windowed
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.28.0/src/windowed.rs:341:9
   9: egui_glow::epi_backend::create_display
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/egui_glow-0.17.0/src/epi_backend.rs:23:9
  10: egui_glow::epi_backend::run
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/egui_glow-0.17.0/src/epi_backend.rs:56:27
  11: eframe::run_native
             at /home/scvalex/.cargo/registry/src/github.com-1ecc6299db9ec823/eframe-0.17.0/src/lib.rs:180:5
  12: sixty_two::main
             at ./src/main.rs:21:5
  13: core::ops::function::FnOnce::call_once
             at /build/rustc-1.56.1-src/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Again, in NixOS, we have to include this manually. In Debian and Fedora, the app finds them without extra work on our part.

We also get this Option::unwrap panic if the system libraries are incompatible with what the app is using. This is why it’s important for the nixpkgs.url to match the NixOS version the system is using.

Wrapping up

This post is long, but the flake itself is fairly short at only 66 lines, which includes several niceties like editor support and prettier diffing.

Like most NixOS things, this was hard to figure out, but once we know how to do it, it’s easy to adapt to different requirements.