Gamlor's Blog: Use Mill Build Caching in CI-Builds

November 5, 2025

Use Mill Build Caching in CI-Builds

Mill build is still my favorite least-hated build tool. (Here is my outdated introduction to it.) In this blog post I show how to use the Mill build to speed up builds on branches.

I am using Bitbucket pipelines in this post. And I’m a newcomer at that =).

You are probably using another CI system, but the concepts most likely apply as well. Jump to the conclusion for a quick summary.

Naive First Build

Let’s assume we have some build that takes a while. For example, in this blog post series I’ve a project with core, shared-utils, domainX-service and ui modules. Each of those has also a test, that take a while to run.

We start of with the most naive build, running ./mill __.test in our CI. Mill is self boots trapping, including JDK etc, so it doesn’t need any special installation (most of the time) or container image.

That allows us in Bitbucket pipelines to use most basic build container and call ./mill -j 2 __.test. The -j 2 sets the parallelism explicitly two 2 concurrent tasks.

bitbucket-pipelines.sh

image: atlassian/default-image:5

pipelines:
  default:
    - step:
        name: Build and Test
        script:
          - './mill -j 2 __.test'

This builds and runs the test fine. However, if we run the build a few times, we notice that it downloads the dependencies on every build:

> ./mill -j 2 __.test
Downloading mill 1.0.6 from https://repo1.maven.org/maven2/com/lihaoyi/mill-dist-native-linux-amd64/1.0.6/mill-dist-native-linux-amd64-1.0.6.exe ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 54.8M  100 54.8M    0     0  98.1M      0 --:--:-- --:--:-- --:--:-- 98.0M
Downloading https://repo1.maven.org/maven2/com/lihaoyi/mill-runner-daemon_3/1.0.6/mill-runner-daemon_3-1.0.6.pom
Downloaded https://repo1.maven.org/maven2/com/lihaoyi/mill-runner-daemon_3/1.0.6/mill-runner-daemon_3-1.0.6.pom
Downloading https://repo1.maven.org/maven2/com/lihaoyi/mill-runner-meta_3/1.0.6/mill-runner-meta_3-1.0.6.pom
Downloading https://repo1.maven.org/maven2/com/lihaoyi/os-lib_3/0.11.5/os-lib_3-0.11.5.pom
# tons more
Downloaded https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.8%2B9/OpenJDK21U-jdk_x64_linux_hotspot_21.0.8_9.tar.gz
============================== __.test ==============================
# The actual build kicking in

Figure 1. Backing up the Internet is Fun!

Cache the Dependency Folder ~./cache

Mill Caches long term dependencies in the ~/.cache directory. (Maybe more directories, depending on the tools used by the build) We want to cache that across builds. Check your CI manual on how you declare that.

In Bitbucket pipelines caches have two parts to then. The directory that is cached, and optionally a set of files that act as key to these caches. The cache is invalidated if these files changed. I use the mill script and the *.mill files as my cache key.

definitions:
  caches:
    home-cache:
      key:
        files:
          # when the script or the build definition changes, new dependencies might be downloaded
          # so, trigger a cache invalidation if these files changed
          - mill
          - "**/*.mill"
      path: ~/.cache
pipelines:
  default:
    - step:
        name: Build and Test
        caches:
          - home-cache
      # Previous script

And TA-DA! The build doesn’t download Maven Central on every run:

> ./mill -j 2 __.test
============================== __.test ==============================
[build.mill-59/64] compile
# build logs

Selective Execution and Build

Mill supports selective execution to speed up builds for branches/pull requests. See the reference and this blog post for more in detail information.

The idea is: Only build and test the parts of the system that actually can be affected by a change. To do that, I wrote a small wrapper script to do the setup:

build-ch.sh

#!/bin/bash
if [[ $BITBUCKET_BRANCH == "main" ]]; then
  # On the main branch, be conservative and build everyting regularly.
  ./mill __.test
else
  # On feature branches, use selective build and testing

  # Ensure we can fetch other branches
  git config remote.origin.fetch "+refs/heads/main:refs/remotes/origin/main"
  git fetch --depth 1 origin

  # Switch to the main branch and let mill determin the state of the source code
  git switch -c main origin/main
  ./mill selective.prepare
  # Switch back to the branch we`re building and build
  # Mill will now will use the build graph and determin what modules can be affected by the difference to the main branch
  git switch $BITBUCKET_BRANCH
  ./mill selective.run __.test
fi

Now, if the main branch is build everything is built and tested:

Main build

# builds and test for all our modules.
# Eg. Mill is running ~650 build tasks in ~94 seconds
[664] Test info.gamlor.ui.customer.CustomerUISlowTest.slowTest4 finished, took 7.001 sec
[664] Test run info.gamlor.ui.customer.CustomerUISlowTest finished: 0 failed, 0 ignored, 4 total, 21.018s
[664/665] ============================== __.test ============================== 94s

Then, for branches it depends on what I touched. Eg. when I only touch the UI module, then less tasks run:

Branch only changing UI

# On a branch: only some UI changed, so many modules and tests got skipped
# So, Mill runs ~230 build tasks in ~38 seconds.
[233] Test info.gamlor.ui.customer.CustomerUISlowTest.slowTest4 finished, took 7.002 sec
[233] Test run info.gamlor.ui.customer.CustomerUISlowTest finished: 0 failed, 0 ignored, 4 total, 21.017s
[1/1] ============================== selective.run __.test ============================== 38s

Branch changing a core module

# On a branch: a more core module changed, now ~610 tasks build in ~65 secons
[610] Test info.gamlor.ui.customer.CustomerUISlowTest.slowTest4 finished, took 7.001 sec
[610] Test run info.gamlor.ui.customer.CustomerUISlowTest finished: 0 failed, 0 ignored, 4 total, 21.016s
[1/1] ============================== selective.run __.test ============================== 65s

This faster build of branches helps to merge smaller pull request faster and have quicker feedback.

Use Mill Caching

Mill caches task results and will skip a task if its upsteam dependencies didn’t change. To accelerate builds for branches more, lets cache the out directory and re-use in for branch builds. The plan is simple: On the main branch build, we skip the cache but store the result in the cache. On the branch builds we use the cache but do not store it.

Figure 2. Little treasure cache named 'out'

This took me a while to figure out in Bitbucket Pipelines:

Plan caches just cache once per week and are too static
Cache invalidation as described above only can take files as keys
I tried to create a step that created a cache-key file dynamically, but the cache key must be committed.

The solution I came up is to run an explicit clear cache command on the main build:

bitbucket-pipelines

definitions:
  caches:
    home-cache:
      # ... like before
    mill-out-cache: out # No cache key, as we manually clear the cache in the script
pipelines:
  default:
    - step:
        name: Build and Test
        caches:
          - home-cache
          - mill-out-cache
        script:
          - ./build-ci.sh

build-ci.sh

#!/bin/bash
if [[ $BITBUCKET_BRANCH == "main" ]]; then
  # Clear the build cache, so it isn't used for the main branch
  curl --request DELETE \
    --url 'https://api.bitbucket.org/2.0/repositories/$BITBUCKET_WORKSPACE/$BITBUCKET_REPO_SLUG/pipelines-config/caches?name=mill-out-cache' \
    --header 'Authorization: Bearer $PIPELINE_TOKEN'
  # Clear the our directory, start with a clear state
  rm -rf out
  ./mill __.test
else
  # ... like before
fi

Cached Tests, alternative to Selected

Now there is caching in place, there is another mechanism that speeds up tests. Mill by default has also the testCached task. Unlike test, it won’t rerun the tests.

So, you could run mill -j __.testCached instead of the selective execution setup we did above.

Even if you are not using it, __.testCached great for local runs, as it doesn’t require extra steps. This way to you can run the test locally and skip modules that are not affected by a chance since the last run.

Summary:

When using Mill in a CI server, then:

Ensure you share/cache the ~/.cache director between builds, as dependencies are cached there.
- Note: Depending on your build, more folder could be used, eg. ~/.npm
To speed up builds for branches/pull requests:
- Try out selective execution/build.
- Consider coping the out directory from the latest main branch build.
The testCached task exists: Great for local builds, you may try it in a CI branch/pull request build as well.

Tags: Scala Mill Build Java Development

Yes, Internet Explorer still runs on Win11, with some help »