Finding things in nixpkgs and NixOS source

Jade Lovelace <jade at jade dot fyi>

Get the slides at https://jade.fyi/nixcon2023/slides

# whoami * Author of nix-doc (not to be confused with nixdoc) * Most involved in Nix for Haskell and Rust packaging * I use NixOS as a dev workstation and server

# the sources are for everyone * We *do* need to improve our documentation * Nix is uniquely participatory: a *lot* of us write modules and packages, which means reading sources * Navigating the source is a bit of witchcraft, and there's a lot of it * With good tooling, the sources of the docs sometimes are faster than the docs * We should have more interactive docs!

# motivation (2.94 million lines of it) <pre><code class="text"><script type="text/template"> dev/nixpkgs » tokei -t nix . ======================================= Language Files Lines ======================================= Nix 30400 2941526 ======================================= Total 30400 2941526 ======================================= </script></code></pre>

# roadmap * nix * static analysis * nixd * ctags * dynamic analysis * using the repl * using nix-doc * nixos * ctags * repl usage

# Static analysis * Any method that takes 15 seconds or less and doesn't involve running any code * First way to try: * Extremely fast * Objective: "works most of the time" * I am showing the fancy ways of doing it here * ripgrep is static analysis too * Objective of fancy ways: faster and less stuff to sort through

# nixd Newish language server, using Nix as a library. In nixpkgs as `nixd`. Good for: * need go-to-definition * interactive help writing nix code Limitations: * does not eval everything * needs some setup per-project * imperfect go-to-definition

# nixd setup Install from nixpkgs and configure in your editor. Then: `.nixd.json`: <pre><code class="json"><script type="text/template"> { "eval": { "target": { "args": [ "--expr", "with import ./. {}; nix-doc" ] } } } </script></code></pre>

# demo (nixd) <video class="r-stretch" src="./nixd.mp4" controls>

# ctags * Very old and simple source code index format * Usually gets through abstraction by being too naive to get broken * Typically generated by simple parse-tree traversal * Wide editor support ([VSCode extension](https://marketplace.visualstudio.com/items?itemName=jaydenlin.ctags-support); also built into vim/emacs) Sample: ```text boot.initrd.luks.reusePassphrases nixos/modules/system/boot/luksroot.nix 533 ```

# ctags on nix * `nix-doc tags .` in nixpkgs, then in vim you can `:tj fixedPoints`, `C-]`, etc. * [WIP ctags for NixOS options (#249243)](https://github.com/NixOS/nixpkgs/pull/249243)

# demo (ctags on nix) <video class="r-stretch" src="./nix-ctags.mp4" controls>

# summary (static analysis) First try language server, then ctags, then ripgrep.

# dynamic analysis * Static analysis often doesn't cut it, power tools time! * `nix repl` is your friend

# using the repl (nix) * `nix repl -f .`, `nix repl -f '<nixpkgs>` * `:lf flake-ref` (more on this later) * `:e derivation-or-function` to open in editor

# getting docs in the repl <pre><code class="text"><script type="text/template"> nix-repl> :doc builtins.head Synopsis: builtins.head list Return the first element of a list; abort evaluation if the argument isn’t a list or is an empty list. You can test whether a list is empty by comparing it with []. </script></code></pre>

# getting docs in the repl (ii) <pre><code class="text"><script type="text/template"> nix-repl> :doc haskell.lib.overrideSrc error: value does not have documentation </script></code></pre> Oops. Related: * [RFC 0145](https://github.com/NixOS/rfcs/pull/145) * [nix#3904](https://github.com/NixOS/nix/issues/3904)

# nix-doc? <pre><code class="nix"><script type="text/template"> { pkgs, ... }: { nix.extraOptions = '' plugin-files = ${pkgs.nix-doc}/lib/libnix_doc_plugin.so ''; } </script></code></pre>

# using nix-doc with the repl <pre><code class="text"><script type="text/template"> nix-repl> builtins.doc haskell.lib.overrideSrc Override the sources for the package and optionally the version. This also takes of removing editedCabalFile. func = drv: src: ... # /nix/store/wl5m5xfayd69ycyspzyd4rilfgl6wmh0-source/pkgs/development/haskell-modules/lib/default.n ix:285 null </script></code></pre>

<video class="r-stretch" src="./dynamic-demo.mp4" controls>

# bonus: using the debugger * pass `--debugger` to nix * use `builtins.break (value)` somewhere in the evaluation path * NOTE: `(v: builtins.break v) value` may get more locals (see [nix#8827](https://github.com/NixOS/nix/issues/8827)) * then use `unsafeGetAttrPos` and others to figure out the source of a value

# summary (dynamic analysis) * `:e function/package` * \+ works pretty well * \+ built-in * \- issues with wrappers (e.g. `fetchFromGitHub`: `lib.makeOverridable`) * \- doesn't help for attr sets * `builtins.doc` (nix-doc) * \+ concise output on functions * \- issues with wrappers

# summary (dynamic analysis) * `builtins.unsafeGetAttrPos` * \+ gets close even with abstraction * \- unergonomic args and return value for interactive use

# NixOS ctags ([PR #249243](https://github.com/NixOS/nixpkgs/pull/249243)) With the PR checked out: <pre><code class="text"><script type="text/template"> $ nix build -f nixos/release.nix optionsCtags -o opts-tags </script></code></pre> Then add `opts-tags` to ctags search path (`:set tags+=opts-tags` in vim).

# NixOS ctags (demo) <video class="r-stretch" src="./nixos-ctags.mp4" controls>

# NixOS in the repl Getting the configuration into the repl: * Non-flakes: `nix repl -I nixos-config=/path/to/configuration.nix -f <nixpkgs/nixos>` * Flakes: `nix repl` then `:lf .` then `nixosConfigurations.xx.{....}`

# NixOS in the repl (demo) <video class="r-stretch" src="./nixos-repl.mp4" controls> <!-- nix-repl> config.documentation.man.generateCaches true nix-repl> options.documentation.man.generateCaches { __toString = «lambda @ /nix/store/wl5m5xfayd69ycyspzyd4rilfgl6wmh0-source/lib/modules.nix:807:22» ; _type = "option"; declarations = [ ... ]; default = false; definitions = [ ... ]; definitionsWith Locations = [ ... ]; description = "Whether to generate the manual page index caches.\nThis allows searching for a page or\nkeyword using utilities like {manpage}`apropos(1)`\nand the `-k` option of \n{manpage}`man(1)`.\n"; files = [ ... ]; highestPrio = 1000; isDefined = true; loc = [ ... ]; opti ons = [ ... ]; type = { ... }; value = true; } nix-repl> options.documentation.man.generateCaches.definitions [ true ] nix-repl> options.documentation.man.generateCaches.definitionsWithLocations [ { ... } ] nix-repl> :p options.documentation.man.generateCaches.definitionsWithLocations [ { file = "/nix/store/wl5m5xfayd69ycyspzyd4rilfgl6wmh0-source/nixos/modules/programs/fish.nix"; va lue = true; } ] -->

# questions? slides: https://jade.fyi/nixcon2023 masto: @leftpaddotpy\@hachyderm.io email: jade at jade dot fyi code: https://github.com/LF- github sponsors: https://github.com/sponsors/LF-