Pinning packages in Nix
Although Nix supposedly makes pinning things easy, it really does not seem so
from a perspective of looking at other software using pinning: it is not
possible to simply write package = "^5.0.1"
in some file somewhere and get
one package pinned at a specific version. Though this is frustrating, there
is a reason for this, and it primarily speaks to how nixpkgs is a Linux
distribution and how Nix is unlike a standard language package manager.
This post will go through the ways to pin a package to some older version and why one would use each method.
Simply add an older version of nixpkgs
Software regressed? No patches in master to fix it? Try 30-40 different versions of nixpkgs. An easy weeknight bug fix. You will certainly not regret pinning 30-40 versions of nixpkgs.
Unlike most systems, it is fine to mix versions of nixpkgs, although it will likely go wrong if, e.g. libraries are intermingled between versions (in particular, it is inadvisable to replace some program with a version from a different nixpkgs from within an overlay for this reason). But, if one package is all that is necessary, one can in fact simply import another version of nixpkgs.
This works because binaries from multiple versions of nixpkgs can coexist
on a computer and simply work. However, it can go wrong if they are loading
libraries at runtime, especially if the glibc version changes, especially if
LD_LIBRARY_PATH
is involved. That failure mode is, however, rather loud and
obvious if it happens.
For example:
let
pkgs1Src = builtins.fetchTarball {
# https://github.com/nixos/nixpkgs/tree/nixos-23.11
url = "https://github.com/nixos/nixpkgs/archive/219951b495fc2eac67b1456824cc1ec1fd2ee659.tar.gz";
sha256 = "sha256-u1dfs0ASQIEr1icTVrsKwg2xToIpn7ZXxW3RHfHxshg=";
name = "source";
};
pkgs2Src = fetchTarball {
# https://github.com/nixos/nixpkgs/tree/nixos-unstable
url = "https://github.com/nixos/nixpkgs/archive/d8fe5e6c92d0d190646fb9f1056741a229980089.tar.gz";
sha256 = "sha256-iMUFArF0WCatKK6RzfUJknjem0H9m4KgorO/p3Dopkk=";
name = "source";
};
pkgs1 = import pkgs1Src { };
pkgs2 = import pkgs2Src { };
in
{
env = pkgs1.buildEnv {
name = "env";
paths = [ pkgs1.vim pkgs2.hello ];
};
vim1 = pkgs1.vim;
vim2 = pkgs2.vim;
}
Here we have an environment which is being built out of packages from two
different versions of nixpkgs, so that result/bin/hello
is from pkgs2
and
result/bin/vim
is from pkgs1
. This can equivalently be done for
environment.systemPackages
or similar such things: to get another version of
nixpkgs into a NixOS configuration, one can:
- For flakes, one can inject the dependency in some manner suggested by
"Flakes aren't real". Or, one can do the
builtins.fetchTarball
thing above. - For non-flakes, one can do the
builtins.fetchTarball
thing shown above, or add another input innpins
/Niv/etc, or add a second channel (though we suggest migrating NixOS configs using channels to npins or flakes so that the nixpkgs version is tracked in git).
» nix-build -A env /tmp/meow.nix
/nix/store/zilav8lqqgfgrk54wg88mdwq582hqdp9-env
~ » ./result/bin/hello --version | head -n1
hello (GNU Hello) 2.12.1
» ./result/bin/vim --version | head -n3
VIM - Vi IMproved 9.0 (2022 Jun 28, compiled Jan 01 1980 00:00:00)
Included patches: 1-2116
Compiled by nixbld
» nix eval -f /tmp/meow.nix vim1.version
"9.0.2116"
» nix eval -f /tmp/meow.nix vim2.version
"9.1.0148"
- Difficulty
- Very easy
- Rebuilds
- None, but will bring in another copy of nixpkgs and any dependencies (and transitive dependencies).
Vendor the package
Another way to pin one package is to vendor the package definition of the
relevant version. The easiest way to do this is to find the version of nixpkgs
with the desired package version and then copy the package.nix
or
default.nix
or such into your own project, and then call it with
callPackage
.
You can find it with something like:
» nix eval --raw -f '<nixpkgs>' hello.meta.position
/nix/store/0qd773b63yg8435w8hpm13zqz7iipcbs-source/pkgs/by-name/he/hello/package.nix:41
Or, equivalently, with nix repl -f '<nixpkgs>'
, :e hello
or to do the same
as above, hello.meta.position
.
Then, vendor that file into your configurations repository.
Once it is vendored, it can be used either from an overlay:
final: prev: {
hello = final.callPackage ./hello-vendored.nix { };
}
or directly in your use site:
{ pkgs, ... }: {
environment.systemPackages = [
(pkgs.callPackage ./vendored-hello.nix { })
];
}
- Difficulty
- Slight effort
- Rebuilds
- For the overlay use case, this will build the overridden package and anything depending on it. For the direct at use site case, this will just rebuild the package, and anything depending on it will get the version in upstream nixpkgs.
Patch the package with overrides
nixpkgs offers several separate methods to "override" things that mean different things. In short:
somePackage.override
replaces the dependencies of a package; more specifically the dependencies injected bycallPackage
. It accepts an attribute set but can also accept a lambda of one argument, providing the previous dependencies of the package.somePackage.overrideAttrs
replaces thestdenv.mkDerivation
arguments of a package. This lets you replace thesrc
of a package, in principle.overrideCabal
replaces thehaskellPackages.mkDerivation
arguments for a Haskell package in a similar way thatoverrideAttrs
does forstdenv.mkDerivation
. This is internally implemented by methods equivalent to the evil crimes below.
Here are some examples:
Build an openttd with a different upstream source by putting this in
openttd-jgrpp.nix
:
{ openttd, fetchFromGitHub }:
openttd.overrideAttrs (old: {
src = fetchFromGitHub {
owner = "jgrennison";
repo = "openttd-patches";
rev = "jgrpp-0.57.1";
sha256 = "sha256-mQy+QdhEXoM9wIWvSkMgRVBXJO1ugXWS3lduccez1PQ=";
};
})
then pkgs.callPackage ./openttd-jgrpp.nix { }
.
For instance, the following (rather silly) command will build such a file:
» nix build -L --impure --expr 'with import <nixpkgs> {}; callPackage ./openttd-jgrpp.nix {}'
Limitations
Most notably, overrideAttrs doesn't work on several
significant language ecosystems including Rust and Go, since one almost always
needs to override the arguments of buildRustPackage
or buildGoPackage
when
replacing something. For these, either one can do crimes to introduce an
overrideRust
function (see below), or one can cry briefly and then vendor the
package. The latter is easier.
let
pkgs = import <nixpkgs> { };
# Give the package a fake buildRustPackage from callPackage that modifies the
# arguments through a function.
overrideRust = f: drv: drv.override (oldArgs:
let rustPlatform = oldArgs.rustPlatform or pkgs.rustPlatform;
in oldArgs // {
rustPlatform = rustPlatform // {
buildRustPackage = args: rustPlatform.buildRustPackage (f args);
};
});
# Take some arguments to buildRustPackage and make new ones. In this case,
# override the version and the hash
evil = oldArgs: oldArgs // {
src = oldArgs.src.override {
rev = "v0.20.9";
sha256 = "sha256-NxWqpMNwu5Ajffw1E2q9KS4TgkCH6M+ctFyi9Jp0tqQ=";
};
version = "master";
# FIXME: if you are actually doing this put a real hash here
cargoSha256 = pkgs.lib.fakeHash;
};
in
{
x = overrideRust evil pkgs.tree-sitter;
}
Then: nix build -L -f evil.nix x
- Difficulty
- Highly variable, sometimes trivial, sometimes nearly impossible, depending on architectural flaws of nixpkgs.
- Rebuilds
- For the overlay use case of actually using this overridden package, this will build the overridden package and anything depending on it. For the direct at use site case, this will just rebuild the package, and anything depending on it will get the version in upstream nixpkgs.
Patch a NixOS module
If one wants to replace a NixOS module, say, by getting it from a later version of nixpkgs, see Replacing Modules in the NixOS manual.
Patch the base system without a world rebuild
It's possible to replace an entire store path with another inside a NixOS
system without rebuilding the world (but wasting some space (by duplicating
things for the rewritten version) and being somewhat evil/potentially unsound
since it is just a text replacement of the hashes). This can be achieved with
the NixOS option
system.replaceRuntimeDependencies
.
Why do we need all of this?
The primary reason that Nix doesn't allow trivially overriding packages with a
different version is that it is a generalized build system building software
that has non-uniform expectations of how to be built. One can indeed see
that the "replace one version with some other in some file" idea is almost
reality in languages that use mkDerivation
directly, though one might have to
tweak other build properties sometimes. Architectural problems in nixpkgs
prevent this working for several ecosystems, though.
Another sort of issue is that nixpkgs tries to provide a mostly globally coherent set of software versions, where, like most Linux distributions, there is generally one blessed version of a library with some exceptions. This is, in fact, mandatory to be able to have any cache hits as a hermetic build system: if everyone was building slightly different versions of libraries, all downstream packages will have different hashes and thus miss the cache.
So, in a way, a software distribution based on Nix cannot have separate locking for every package and simultaneously have functional caches: the moment that everything is not built together, caches will miss.