I found something mysterious while working on a large Haskell codebase: there was a Cabal target called test:test-dev, containing test and src in source-dirs. For a while, I didn't realise what it's for, and it seemed like the kind of thing that someone goes and creates for some old use case. Now I have put this into at least three other codebases because it fixes my problems. What changed?

I learned about some limitations of cabal, GHC, and GHCi:

What if you could Not Have Those Problems? It turns out, they are all solved by Putting More Code in a Target, although sometimes this needs to be done with a little more persuasion of the build system.

If you think you have too much code in one package, you are probably mistaken. Most of GHC, representing a Lot of Haskell, is one package. Avoiding the limitations of the build infrastructure around multiple packages improves build times and developer productivity.

How do I test-dev?

Include both src and test in hs-source-dirs (source-dirs in hpack), and then use cabal run test:test-dev (yes, run; cabal's shiny new test runner doesn't let you control the entry point, which is no fun), or cabal repl test:test-dev.

You will need to put all your dependencies for src/ and your dependencies for test/ in the dependencies for test-dev.

But what if a dependency is causing cross-target sadness?

In the spirit of the yolo method of setting up HLS in which you unceremoniously temporarily stuff some third party project into a cabal workspace with your code, you can also just lie to cabal that the dependency is part of your package.

Simply tell cabal "it's my package now" by putting its source directory into hs-source-dirs and remove the dependency (since it's your package now). Cabal will apparently not think much of it!

Let's use hspec as an example. First clone hspec below your project directory (nested git repos are no problem as long as they remain untracked):

$ git clone https://github.com/hspec/hspec

Then do something like this in your cabal file:

test-suite test-dev
    -- ...
    hs-source-dirs:
        src
        test
        hspec/hspec
        hspec/hspec-core
    build-depends:
        -- your app depends PLUS (your test depends MINUS hspec/hspec-core) PLUS hspec/hspec-core's depends

Finally you can cabal run test:test-dev, or cabal repl test:test-dev, and it will build your code and the hspec code all as one imaginary package, allowing you to use the full GHCi feature set and compile faster, especially if you're actively working on hspec.

In this way, you can blur the line between dependencies and your own code, working on them as one, while also keeping them separate the rest of the time.

Bonus section: It also makes HLS work better

You can also put test-dev in place of the other targets in hie.yaml in all the paths it includes, which will improve HLS's overall performance and usability, although perhaps at the cost of more startup time. This is because HLS has the same problem as cabal where it seemingly doesn't build dependent targets in parallel with their dependencies.

More frustratingly, if a dependency fails to compile, it will sometimes take down all the dependent code with it, even if the dependent code doesn't all actually import the dependency.

You can set this up in hie.yaml something like so:

cradle:
  cabal:
    - path: "src"
      component: "test:test-dev"

    - path: "test"
      component: "test:test-dev"

Are split packages a good idea in some contexts?

Here's a blog post by Matt Parsons on compile times, which I generally agree with based on my experience working in large Haskell codebases.

Some highlights include:

Given a package B, depending on A:

By combining A and B into a single package, we sped up compile times for a complete build of the application by 10%. A clean build of the new AB package was 15% faster to build all told, and incremental builds were improved too.

He suggests "if you're not willing to GitHub it, then it should probably stay in the main package".

I agree with this heuristic: you can cache the entire thing as a package (in fact, Nix will automatically do it for you) if it's on GitHub/Hackage, and it probably doesn't change that often if you are willing to do that, so the slight increase in annoyingness of developing it is probably fine.

That said, you can have your cake and eat it too! It's easily possible to stuff dependencies into your workspace while working on them, or even artificially integrate them into your own package while working on them via the test-dev trick.

Using these methods, you can get exactly the same developer experience while working on libraries as if they are fully part of the codebase by telling cabal that they are fully part of the codebase only while developing on them, then have them be separate while releasing.


The elephant in the room: "Wait, GHC fixed this?"

My post describes a workaround for the absence of a GHC feature called "multiple home units", which released in GHC 9.4, and is supposed to solve these problems. However, Cabal does not yet support it, which is rather a roadblock. Also, GHCi has limited functionality under multiple home units.

You can read more about the ongoing work on multiple home units at the Well-Typed blog post on the subject.

That post points out that Stack does the test-dev hack described in this article if you do something like stack repl exe:myexample lib:myexample, but I don't use Stack.

The test-dev trick will continue working into the future, but I look forward to the day that it is no longer necessary.

Limitations

This hack has some unfortunate limitations, many of which are discussed in the posts about multiple home units above:

Bonus section: ghcid on tests

You know what fixing GHCi on tests in the presence of app modifications does? That's right, you can run your test suite with ghcid on every file save. This is seriously awesome since it uses interpreted mode for those sweet GHCi reload times.

This section was inspired by Matt Parsons' blog post on ghcid, which incidentally uses this trick because Stack does it under the hood!

Back to our regularly scheduled bonus content:

First, create a file test/Main.hs something like so (this circumvents the cabal test runner which doesn't let you have a custom entry point):

module Main where
import Spec (spec)
import Hspec
main :: IO ()
main = testMain
-- avoids overlapping names with Spec.main
testMain :: IO ()
testMain = hspec spec

Then you can ghcid your tests with:

$ ghcid --command 'cabal repl test:test-dev --ghc-options="-osuf dyn_o -hisuf dyn_hi"' --test testMain

When you save any file, ghcid will pilot the GHCi session to reload and rerun the tests.

Acknowledgements

Thanks to Hazel Weakly, Matt Parsons, and Chris Zehner for their valuable feedback and discussion on drafts of this post.