Building Nix derivations manually in nix-shell

When writing, debugging, or otherwise working with Nix expressions, it is often useful to run a build part of the way through, or to run it manually in a shell. For instance, perhaps you want to test some development version of a tool against a package and want to iterate quickly. Or, a build is broken and you want to look at it more closely than nix-build --keep-failed makes convenient.

Most packages are built with the generic builder, regardless of language. The language specific builders are then written in terms of it. For instance, Haskell packages are built with this builder, customizing it by adding setupCompilerEnvironmentPhase and overriding many phases.

The way nix-build typically builds packages with the generic builder is documented in the stdenv chapter of the nixpkgs documentation. More or less, it runs various "phases" in order to build the package. This is roughly analogous to the build(), check(), etc functions in an Arch Linux or Alpine Linux PKGBUILD file.

The generic builder is made of function definitions in the file pkgs/stdenv/generic/setup.sh in the nixpkgs repository, and it is usually sourced by the actual build script, pkgs/stdenv/generic/default-builder.sh, which is trivial:

source $stdenv/setup
genericBuild

default-builder.sh is not always used: setup.sh is more or less a shell script library and can equally be used in a custom build script that defines some functions then calls genericBuild.

By default, genericBuild will run the default phases, the most notable of which are unpackPhase, configurePhase, buildPhase, checkPhase (tests), and installPhase. There are also hooks that can run. These phases are either the functions of that name, which perform the default C ./configure; make; make install thing, environment variables containing scripts overriding them, or shell functions defined in the file that sources setup.sh.

Another feature supported by the generic builder is "hooks" which are basically little shell functions, possibly defined by environment variables, which can be run at various points in the build process if present.

More phases can also be added for more complicated things to build. In practice, all of these extensibility features mean that the generic builder is reused for most language ecosystems.

Running a build manually

Because of the complication of possible custom phases, hooks, etc, it is not really prudent to just run the phase functions directly because it essentially means you are reimplementing genericBuild in your head.

When debugging, what you probably want to do is consult the full phases list and run genericBuild with those phases. We can see that the default phases are the following (from setup.sh):

if [ -z "${phases:-}" ]; then
    phases="${prePhases:-} unpackPhase patchPhase ${preConfigurePhases:-} \
        configurePhase ${preBuildPhases:-} buildPhase checkPhase \
        ${preInstallPhases:-} installPhase ${preFixupPhases:-} fixupPhase installCheckPhase \
        ${preDistPhases:-} distPhase ${postPhases:-}";
fi

A manual build of a package might look like this:

First, open a nix-shell with the derivation of the package in question. For instance, nix-shell '<nixpkgs>' -A hello.

Then do:

phases="${prePhases:-} unpackPhase patchPhase" genericBuild

which will dump you into a shell in the sources directory for the package.

Once you've done the stuff you'll run once, you can run the build as many times as you'd like:

out=$(pwd)/out phases="${preConfigurePhases:-} configurePhase ${preBuildPhases:-} buildPhase" genericBuild

The $out variable has to be specified because it defaults to somewhere in the nix store, which usually breaks builds since is not available for writing if you are not the actual sandboxed nix builder. If there are other outputs such as doc, these also need to be specified here.