NixCon 2022 retrospective

Well, that sure was a thing. It was more than a thing: it was a really good thing.

NixCon 2022 was my first in-person conference, and therefore also my first time speaking at an in-person conference. I understand better why people want these. It's really nice to be around The People For The Thing and meet new people (whether they are wearing cat ears or otherwise).

I met a whole bunch of people and made some very good friends.

Speaking - nix-closure-graph

I delivered a talk "Debugging Closure Size Graphically". This was a fun experience; it's a lot easier to deliver jokes in front of an audience. I'm excited to see what people do with the tools I built, and I'm glad they are open source now.

You can get more about it on the talk page.

The hacking room

I spent most of the conference either in breakout rooms and later, the hackathon room. We got the following done:

Incremental builds for Haskell

This was done after an extremely useful discussion with Jonas Chevalier (@zimbatm) and implemented in large part by Harry Garrood. The idea is that we can achieve incremental builds of Haskell by doing an impure dependency on the previous build, intentionally breaking the evaluation chain.

You can get an example project showing what we developed from Harry's GitHub.

In practice, this looks like a setting on the Haskell builder to enable a separate output for incremental information (which is the interface and object files of the run), and a second setting to copy in incremental information from some path.

This approach requires GHC 9.4 in order to use hash-based incremental information rather than entirely relying on file timestamps as was previously done.

The reason this is fantastic is that it avoids the following false dichotomy of flawed options:

• Use one derivation per file. This would pessimize build times by having to invoke GHC for each file as ghc-nix does, eating startup costs for each file, and also losing finer grained incremental support than "did the file change".

  That is a completely valid and reasonable approach as it keeps the build hermetic at the cost of more build time, but it relies heavily on content-addressed derivations, and if you used it within Nix, it would also use recursive Nix or derivations-building-derivations.

• Change the project structure to use more packages so that Nix can incrementally build at the package level.

  This is hard to do, and multi-package pessimizes the developer experience due to lack of multiple home units support as well as slowing down builds in development (as opposed to Nix builds) since Cabal/GHC do not yet know how to build the non-dependent parts of dependent packages at the same time.

  In my view, commercial projects probably should be one package as it currently stands, because splitting packages is bad for development efficiency, requiring duplicating dependency lists and doing a bunch of other housekeeping, on top of the issues with concurrently working on multiple packages described above.

  Note that as in the link above, calling GHC once per file does allow solving the dependent-package parallelism problem, as Bazel does, but it also pessimizes build times.

We decided to have our cake and eat it too. We prototyped an approach of letting GHC do the incremental builds in the way Harry describes, and then convincing Nix to let us do it. This leaves dev completely alone, and constitutes only a minor impurity crime (since at least it is reproducible given some effort!), while not pessimizing compile times at all.

nix-otel grows up

Another really nice achievement of the conference is that, with the help of Linus Heckemann and Jean-François Roche, we have improved data quality in nix-otel and poked at making it work in the daemon, after which it could be integrated into Nix itself.

Following discussions, it's likely that future structured logging will be OpenTelemetry based as it avoids reinventing the wheel.

The data quality improvement is that previously, changing phases in the builder would not emit spans, which it now does. There is one span for each phase. This was achieved by effectively postprocessing the log data from Nix. Also, there was a memory corruption I fixed, which was somehow only causing missing data rather than crashes. Yikes.

We also now report 100% of the information that Nix gives us via the logger, which means that further improvements will be in Nix itself, improving logging for everyone.

Linus worked on getting nix-otel to work in the daemon, which is still a work in progress. We want to use settings for the API keys and endpoint to avoid needing environment variables for them. On that account, we arrived at hunting a bug in our settings handling in which the settings were not getting their values properly.

Another thing that was discussed is making the daemon and client cooperate since the client knows some things the daemon doesn't, and the daemon has better timing information. This can be done pretty easily by having the client do propagation of trace IDs to the daemon, and keeping track of whether log data was forwarded (and in that case, not sending it to OpenTelemetry since it already was sent).

Once this foundational work is done, nix-otel can be integrated into Nix itself, potentially representing the first Rust in Nix since the previous attempt that had significantly more complicated foreign-function-interface usage, leading to its failure.

Looking forward, another thing we can probably do is to use the $NIX_LOG_FD infrastructure that already exists to build an OpenTelemetry propagator/exporter that can be used within builds to also instrument the inside of builds.

oops, feelings

Paris is beautiful. I really appreciate having a practical transit system and being able to walk everywhere.

Certainly this trip has renewed my motivation for looking into possibly moving to Europe in the next few years.

Thanks to Harry Garrood for reviewing a draft of this post.