Another Haskell and Nix setup

Note: after reading this post, read this Reddit post by a Nixpkgs Haskell maintainer who has many good suggestions for further improvement.

There are already several guides on how to build Haskell programs with Nix out there. However, when I set up Nix-based builds for a Haskell program I maintain, the examples I could find were either too simple, or far too complicated. I’m not very skilled at using Nix, so I needed concrete examples I could understand and modify. Eventually I cobbled together something I’m mostly happy with, which I’m now writing up as the example I would have liked to have found myself. What follows is a superficial walkthrough of the files I use to build tarballs containing binary builds of the Futhark compiler.

• futhark.cabal is an ordinary Cabal file. Nothing special here.

• futhark.nix is generated from futhark.cabal with cabal2nix . >futhark.nix. It contains only a subset of the information in the Cabal file, most importantly the Haskell-level dependencies. I regenerate it manually whenever I change the dependencies in the Cabal file. Since it is not modified manually, I suppose I could generate it when needed instead.

• nix/sources.nix and nix/sources.json are generated by niv and pin the version of Nixpkgs that everything is built against. This is very important, as otherwise nix-build will build against whatever version of Nixpkgs happen to be available on the system, ruining any hope of reproducibility. Whenever I wish to bump the version of Nixpkgs I’m using, I run niv update nixpkgs -b nixpkgs-unstable, which will update nix/sources.json (niv will tell you when and how to regenerate sources.nix).

• nix/versions.nix contains a Nix derivation for a version of the Haskell package versions for which the derivation in Nixpkgs itself is too old. It is generated by cabal2nix cabal://versions-5.0.0 > nix/versions.nix, and the resulting .nix file must be manually imported elsewhere (see below).

• default.nix is the “main” Nix file. It imports futhark.nix, sources.nix, and versions.nix and uses these to define a Nix derivation that builds the statically linked Futhark binaries. Additionally, it contains postBuild and postInstall stages to build the manpages (which requires Sphinx). This derivation is then used as a dependency for the actual top-level derivation, which takes the generates files and puts them together in a tarball. This means that nix-build produces a result/ directory that contains a tarball futhark-nightly-linux-x86_64.tar.xz. The nightly part can be replaced by running nix-build --argstr suffix foo, which will use foo as the suffix. We use this in CI to distinguish “nightly” tarballs from tarballs corresponding to actual releases.

• shell.nix is not used for nix-build, but listed for the sake of completeness. It includes nix/sources.nix to ensure that nix-shell uses the same Nixpkgs as nix-build. This means the same version of GHC, but not Haskell library dependencies. If you run cabal or stack inside a nix-shell, then they will resolve dependencies in the usual way.

Remaining issues

I am mostly happy with this setup, but it does have problems. One is that I link statically with glibc - this is usually considered a bad idea, because glibc’s name lookup service depends on dynamic linking. The program we’re building doesn’t perform any network requests, so this does not matter to us, but it might matter to others.

Another problem is that the program is built against the Haskell packages present in the pinned version of Nixpkgs. This usually corresponds to a recent LTS snapshot from Stackage, which is nice in the sense that everything is internally compatible, but has two major problems:

1. Stackage LTS can move frustratingly slowly, because they cannot update to new versions until all users have catched up. You might need newer versions than what Nixpkgs contains, in which case you need to use cabal2nix to define a derivation for a newer version, just like I do for versions above.

2. It is unlikely that the versions of library dependencies included in Nixpkgs will be the exact same versions as you’d get from a stack build or cabal build (whose solvers don’t use Nixpkgs). This means that nix-build will use different dependencies than you will be using during your own development. I have not been bitten by this yet, but I imagine it can result in an interesting debugging experience.

A better solution would be to run the Cabal solver against Hackage and generate a .nix file for exactly those versions that it picks. I don’t know how to do that, but it seems like an obviously good idea, so I expect that cabal2nix supports it somehow.

A supplementary rant

I think putting Haskell libraries in Nixpkgs is a fundamentally bad idea. The philosophy that every Haskell package in Nixpkgs must use the same versions of library dependencies is not scalable, and in practice often means that the futhark derivation in Nixpkgs is broken. This doesn’t affect my own CI builds (I use cabal2nix to supplement Nixpkgs with whatever I need), but it does mean that I cannot in good faith tell NixOS users to install Futhark from their package system - the odds that it will be broken at any given time are far too high. I’m sure Nixpkgs is careful to ensure major Haskell programs such as Pandoc and ShellCheck remain working, usually by including multiple versions of problematic dependencies in ad-hoc ways, but smaller Haskell programs seem to be out of luck. A better solution would be to give every Haskell program its own distinct set of Haskell library dependencies, resolved by Cabal (or Stack).