mirror of
https://github.com/coder/coder.git
synced 2026-06-02 20:48:20 +00:00
refactor: build dogfood image as base + mise oci layers (#25448)
Splits the dogfood image into two artifacts: - `ghcr.io/coder/oss-dogfood-base:<distro>-<base-sha>`: Ubuntu base with apt packages, chrome, rustup, brew, gh, and the mise binary. The base-sha is a cache key over `Dockerfile.base` and `files/`, so commits that don't touch those inputs reuse the previous build. - `codercom/oss-dogfood:<final-sha>-<distro>` and rolling tags (`:22.04`, `:26.04`, `:latest`, `:<branch>`): produced by `mise oci build` on top of the base, with one content-addressed OCI layer per mise tool. The rolling tag scheme is unchanged, so the workspace template doesn't need updating. Single-tool version bumps now invalidate only that tool's OCI layer, so workspaces re-pull just what changed instead of the entire 5-6 GB image on every recreate. Also: - Drops the build-time `pnpm dlx playwright@1.47.0 install --with-deps chromium` step (~400 MB) and the equivalent `playwright-driver.browsers` install from `flake.nix`. `@playwright/mcp` (used by the claude-code and codex MCP servers in `dogfood/coder/main.tf`) does NOT auto-install browsers, so the existing `install-deps` `coder_script` now runs two installs on workspace start: `pnpm exec playwright install chromium` for the site's pinned `@playwright/test`, and `npx --package=@playwright/mcp@latest playwright-core install --no-shell chromium` so the MCP servers find their matching browser revision. Browser revisions coexist under `~/.cache/ms-playwright/chromium-<rev>/`, which lives on the home volume so both downloads happen once per workspace recreate and persist across restarts. Net effect: same MCP behavior as before, +~1-2 min on first workspace start. Nix devshell users running site e2e tests locally now need `pnpm exec playwright install` once (instead of getting browsers via nixpkgs). - Bumps the pinned mise binary to v2026.5.12 (matching main after #25521) and adds top-level `min_version = "2026.5.12"` to `mise.toml` so every consumer (devs, CI, the embedded mise inside the dogfood image, mise oci builds) fails fast on an older mise. - Adds bison, flex, libicu-dev, libreadline-dev, uuid-dev, and zlib1g-dev to both Ubuntu base images for source-build use cases (e.g., building Postgres from source). - Replaces skopeo with crane as the registry client `mise oci push` shells out to: crane is added to `mise.toml`, the workflow drops its `apt-get install skopeo` and forces `--tool crane`, and the local wrapper image stops bundling skopeo. One source of truth for tool versions, no apt drift, smaller wrapper image, and workspace users get a registry client on PATH for free via mise oci's tool layers. - Removes `nix.hash`/`mise.hash` and their Makefile rules. The registry digest already captures every effective change since CI rebuilds when any baked-in input moves; the per-file `filesha1()` entries in `pull_triggers` are redundant. Supersedes #25400 (the `mise.hash` pull trigger landed there in `2b612abe7b`; this PR removes it as part of the broader simplification). > [!NOTE] > `mise oci build` is experimental and requires `MISE_EXPERIMENTAL=1` (set at job level in the workflow). The local-only `scripts/dogfood/mise-oci-wrapper.sh` builds a tiny `coderdev/mise-oci-wrapper:<version>` Debian image with curl-installed mise on first invocation (cached by version tag thereafter); we don't reuse `jdxcode/mise:latest` because that tag lags upstream GitHub releases by days and would defeat the `min_version` enforcement above. > [!NOTE] > `compute-base-sha.sh` and `compute-final-sha.sh` are cache keys, not strict content addresses: the base Dockerfile still pulls dynamic resources at build time (gh/buildx `releases/latest`, chrome `stable_current_amd64.deb`, apt mirror state). Two runs with identical checked-in files can produce slightly different bytes, which is acceptable here because the cache-hit savings on irrelevant commits outweigh that drift. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Thomas Kosiewski <tk@coder.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Executable
+43
@@ -0,0 +1,43 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deterministic 12-char content hash of base-image inputs for a distro.
|
||||
# Used as a cache key for the ghcr.io/coder/oss-dogfood-base tag so
|
||||
# commits that don't touch the base inputs reuse the previous build.
|
||||
#
|
||||
# This is NOT a strict content address: the base Dockerfile still
|
||||
# pulls dynamic resources at build time (gh/buildx releases/latest,
|
||||
# chrome stable_current_amd64.deb, apt mirror state, sh.rustup.rs).
|
||||
# Two runs with identical checked-in files can still produce slightly
|
||||
# different bytes. That's acceptable here because the dynamic drift
|
||||
# is small and the cache-hit savings (no full base rebuild for a
|
||||
# typo-fix commit, doc change, mise.toml bump, etc.) is large.
|
||||
set -euo pipefail
|
||||
|
||||
# 12 hex chars matches docker/OCI short-digest displays.
|
||||
HASH_LEN=12
|
||||
|
||||
distro="${1:?usage: $0 <22.04|26.04>}"
|
||||
|
||||
repo_root="$(git rev-parse --show-toplevel)"
|
||||
cd "$repo_root"
|
||||
|
||||
paths=(
|
||||
"dogfood/coder/ubuntu-${distro}/Dockerfile.base"
|
||||
"dogfood/coder/ubuntu-${distro}/files"
|
||||
)
|
||||
if [ "$distro" = "22.04" ]; then
|
||||
paths+=("dogfood/coder/ubuntu-${distro}/configure-chrome-flags.sh")
|
||||
fi
|
||||
|
||||
# Skip editor turds; .swp / ~-files / dotfiles are noise for a build
|
||||
# hash. Include symlinks too: `COPY dogfood/coder/ubuntu-*/files /`
|
||||
# bakes their target paths into the image, so swapping a symlink
|
||||
# changes base content and must invalidate the cache key.
|
||||
find "${paths[@]}" \( -type f -o -type l \) \
|
||||
! -name '.*' \
|
||||
! -name '*.swp' \
|
||||
! -name '*~' \
|
||||
-print0 |
|
||||
LC_ALL=C sort -z |
|
||||
xargs -0 sha256sum |
|
||||
sha256sum |
|
||||
cut -c"1-$HASH_LEN"
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deterministic 12-char content hash of (base inputs + mise inputs) for
|
||||
# a distro. Used as the primary tag for the dogfood image produced by
|
||||
# `mise oci build`, so re-running CI on an unchanged commit reuses the
|
||||
# previous tag. Same cache-key (not strict content address) semantics
|
||||
# as `compute-base-sha.sh`.
|
||||
set -euo pipefail
|
||||
|
||||
# 12 hex chars; see comment in compute-base-sha.sh.
|
||||
HASH_LEN=12
|
||||
|
||||
distro="${1:?usage: $0 <22.04|26.04>}"
|
||||
|
||||
repo_root="$(git rev-parse --show-toplevel)"
|
||||
cd "$repo_root"
|
||||
|
||||
base_sha="$("$repo_root/scripts/dogfood/compute-base-sha.sh" "$distro")"
|
||||
mise_hash="$(sha256sum mise.toml mise.lock | sha256sum | cut -c"1-$HASH_LEN")"
|
||||
|
||||
printf '%s\n' "$base_sha-$mise_hash" | sha256sum | cut -c"1-$HASH_LEN"
|
||||
Executable
+109
@@ -0,0 +1,109 @@
|
||||
#!/usr/bin/env bash
|
||||
# Local-only helper: runs `mise oci ...` inside a Linux container so
|
||||
# macOS and Windows developers don't need a local Linux VM or a host
|
||||
# install of mise. CI runs `mise oci` directly on its Linux runner; it
|
||||
# does not use this script.
|
||||
#
|
||||
# Builds a small Debian-based wrapper image with the mise binary on
|
||||
# first invocation, then reuses it. Pinning to the same `MISE_VERSION`
|
||||
# baked into `Dockerfile.base` avoids depending on jdxcode/mise Docker
|
||||
# Hub publication cadence, which lags upstream GitHub releases by days.
|
||||
#
|
||||
# `oci build --from <ref>` requires <ref> to be a registry-resolvable
|
||||
# reference; the host's local Docker daemon images are not visible
|
||||
# inside the wrapper. See the Makefile comment.
|
||||
#
|
||||
# Honors CONTAINER_RUNTIME=docker (default) or CONTAINER_RUNTIME=container
|
||||
# (Apple's `container` CLI on macOS).
|
||||
set -euo pipefail
|
||||
|
||||
# Keep MISE_VERSION + MISE_SHA256 in lockstep with the same vars in
|
||||
# .github/workflows/dogfood.yaml and dogfood/coder/ubuntu-*/Dockerfile.base.
|
||||
# A `min_version` check in mise.toml catches downgrades.
|
||||
MISE_VERSION="v2026.5.12"
|
||||
MISE_SHA256="a238972a3162d710b85b28c324372e96ca4e4b486c81fe78695000d9fbc77c48"
|
||||
# Bump the -rN suffix when the Dockerfile heredoc below changes
|
||||
# (mise version, apt packages, trust config, etc.) so cached wrapper
|
||||
# images get rebuilt automatically.
|
||||
WRAPPER_REVISION="r2"
|
||||
RUNTIME="${CONTAINER_RUNTIME:-docker}"
|
||||
WRAPPER_IMAGE="coderdev/mise-oci-wrapper:$MISE_VERSION-$WRAPPER_REVISION"
|
||||
|
||||
# Mount the repo root rather than $PWD: `make -C dogfood/coder` invokes
|
||||
# the wrapper from dogfood/coder/, but the project mise.toml/mise.lock
|
||||
# `mise oci build` consumes live at the repo root.
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||
|
||||
platform_arg=()
|
||||
if [ "$RUNTIME" = "container" ]; then
|
||||
platform_arg=(--platform linux/amd64)
|
||||
fi
|
||||
|
||||
# Build the wrapper image on first invocation. The tag includes the
|
||||
# mise version so a bump automatically invalidates the cache; the old
|
||||
# image becomes orphaned and the user can prune it manually.
|
||||
if ! "$RUNTIME" image inspect "$WRAPPER_IMAGE" >/dev/null 2>&1; then
|
||||
echo "[$0] Building $WRAPPER_IMAGE (first-time setup)..." >&2
|
||||
build_dir="$(mktemp -d)"
|
||||
trap 'rm -rf "$build_dir"' EXIT
|
||||
cat >"$build_dir/Dockerfile" <<DOCKERFILE
|
||||
FROM debian:bookworm-slim
|
||||
# crane (the registry client mise oci shells out to) is installed via
|
||||
# mise.toml at run time, not here. Keeps the image lean and avoids
|
||||
# version drift between this base layer and what mise oci uses.
|
||||
RUN apt-get update -qq && \\
|
||||
apt-get install -y -qq --no-install-recommends \\
|
||||
ca-certificates curl && \\
|
||||
rm -rf /var/lib/apt/lists/* && \\
|
||||
curl -sSLf "https://github.com/jdx/mise/releases/download/${MISE_VERSION}/mise-${MISE_VERSION}-linux-x64" -o /usr/local/bin/mise && \\
|
||||
echo "${MISE_SHA256} /usr/local/bin/mise" | sha256sum -c && \\
|
||||
chmod +x /usr/local/bin/mise && \\
|
||||
install --directory --mode=0755 /etc/mise /etc/mise/conf.d && \\
|
||||
printf '[settings]\\ntrusted_config_paths = ["/src"]\\n' > /etc/mise/conf.d/00-trust.toml
|
||||
DOCKERFILE
|
||||
"$RUNTIME" build ${platform_arg[@]+"${platform_arg[@]}"} -t "$WRAPPER_IMAGE" "$build_dir"
|
||||
rm -rf "$build_dir"
|
||||
trap - EXIT
|
||||
fi
|
||||
|
||||
token_arg=()
|
||||
if [ -n "${GITHUB_TOKEN:-}" ]; then
|
||||
token_arg=(-e "GITHUB_TOKEN=$GITHUB_TOKEN")
|
||||
fi
|
||||
|
||||
# Mount ~/.docker when present so crane can find registry creds.
|
||||
# Apple `container` CLI users without Docker Desktop won't have it;
|
||||
# local builds don't push, so the skip is fine.
|
||||
docker_config_arg=()
|
||||
if [ -d "$HOME/.docker" ]; then
|
||||
docker_config_arg=(-v "$HOME/.docker:/root/.docker:ro")
|
||||
fi
|
||||
|
||||
# `oci build` needs all mise tools installed so it can package them
|
||||
# into layers. `oci push` needs crane on PATH (mise oci shells out to
|
||||
# it). Both end up running `mise install` first; build installs every
|
||||
# tool, push only crane. The `export PATH=...` exposes mise's shims
|
||||
# dir so `which crane` succeeds when mise oci spawns it as a child.
|
||||
# Single quotes are intentional: $HOME and $@ expand inside the
|
||||
# container's `sh -c`, not in this script.
|
||||
# shellcheck disable=SC2016
|
||||
inner_cmd='mise oci "$@"'
|
||||
case "${1:-}" in
|
||||
build)
|
||||
# shellcheck disable=SC2016
|
||||
inner_cmd='mise install --yes && export PATH="$HOME/.local/share/mise/shims:$PATH" && mise oci "$@"'
|
||||
;;
|
||||
push)
|
||||
# shellcheck disable=SC2016
|
||||
inner_cmd='mise install --yes crane && export PATH="$HOME/.local/share/mise/shims:$PATH" && mise oci "$@"'
|
||||
;;
|
||||
esac
|
||||
|
||||
exec "$RUNTIME" run --rm ${platform_arg[@]+"${platform_arg[@]}"} \
|
||||
-v "$REPO_ROOT":/src -w /src \
|
||||
${docker_config_arg[@]+"${docker_config_arg[@]}"} \
|
||||
-e MISE_EXPERIMENTAL=1 \
|
||||
${token_arg[@]+"${token_arg[@]}"} \
|
||||
--entrypoint /bin/sh \
|
||||
"$WRAPPER_IMAGE" \
|
||||
-c "$inner_cmd" -- "$@"
|
||||
@@ -50,17 +50,21 @@ else
|
||||
fi
|
||||
|
||||
# Helper: run a make target inside the image.
|
||||
# Caches are persisted in named Docker volumes so that subsequent steps (and
|
||||
# repeated local runs) reuse downloaded modules and compiled artifacts.
|
||||
#
|
||||
# Mounts /home/coder/ as a single named volume to mirror the dogfood
|
||||
# workspace template (dogfood/coder/main.tf), so caches (Go modules,
|
||||
# Go build, pnpm store, mise data, etc.) persist the same way they do
|
||||
# in real workspaces. Per-cache subpath volumes would come up
|
||||
# root-owned on first mount because Docker creates non-existent
|
||||
# subpaths root-owned; the home-level volume inherits coder:coder
|
||||
# from the image's existing /home/coder (`useradd --create-home`).
|
||||
run_make() {
|
||||
docker run --rm \
|
||||
--volume coder-dogfood-home:/home/coder \
|
||||
--volume "$(pwd)":/home/coder/coder \
|
||||
--env GIT_CONFIG_COUNT=1 \
|
||||
--env GIT_CONFIG_KEY_0=safe.directory \
|
||||
--env GIT_CONFIG_VALUE_0=/home/coder/coder \
|
||||
--volume coder-dogfood-gomod:/home/coder/go/pkg/mod \
|
||||
--volume coder-dogfood-gobuild:/home/coder/.cache/go-build \
|
||||
--volume coder-dogfood-pnpm:/home/coder/.local/share/pnpm/store \
|
||||
--workdir /home/coder/coder \
|
||||
--network=host \
|
||||
--env GITHUB_TOKEN \
|
||||
|
||||
@@ -37,6 +37,4 @@ echo "protoc-gen-go version: $PROTOC_GEN_GO_REV"
|
||||
PROTOC_GEN_GO_SHA256=$(nix-prefetch-git https://github.com/protocolbuffers/protobuf-go --rev "$PROTOC_GEN_GO_REV" | jq -r .hash)
|
||||
sed -i "s#\(sha256 = \"\)[^\"]*#\1${PROTOC_GEN_GO_SHA256}#" ./flake.nix
|
||||
|
||||
make dogfood/coder/nix.hash
|
||||
|
||||
echo "Flake updated successfully!"
|
||||
|
||||
Reference in New Issue
Block a user