Commit Graph

14 Commits

Author SHA1 Message Date
Steven Masley 60b3fd0783 chore!: send modules archive over the proto messages (#21398)
# What this does

Dynamic parameters caches the `./terraform/modules` directory for parameter usage. What this PR does is send over this archive to the provisioner when building workspaces.

This allow terraform to skip downloading modules from their registries, a step that takes seconds. 

<img width="1223" height="429" alt="Screenshot From 2025-12-29 12-57-52" src="https://github.com/user-attachments/assets/16066e0a-ac79-4296-819d-924f4b0418dc" />


# Wire protocol

The wire protocol reuses the same mechanism used to download the modules `provisoner -> coder`. It splits up large archives into multiple protobuf messages so larger archives can be sent under the message size limit.

# 🚨  Behavior Change (Breaking Change) 🚨 

**Before this PR** modules were downloaded on every workspace build. This means unpinned modules always fetched the latest version

**After this PR** modules are cached at template import time, and their versions are effectively pinned for all subsequent workspace builds.
2026-01-09 11:33:34 -06:00
Steven Masley 89f4d60e7b chore: remove experiment "terraform-directory-reuse" (#21397)
Experiment is no longer required, the new method will be released without an experiment and without a toggle

Main PR is: https://github.com/coder/coder/pull/21398
2026-01-09 11:13:16 -06:00
Spike Curtis bddb808b25 chore: arrange imports in a standard way (#21452)
Fixes all our Go file imports to match the preferred spec that we've _mostly_ been using. For example:

```
import (
	"context"
	"time"

	"github.com/prometheus/client_golang/prometheus"
	"golang.org/x/xerrors"
	"gopkg.in/natefinch/lumberjack.v2"

	"cdr.dev/slog/v3"
	"github.com/coder/coder/v2/codersdk/agentsdk"
	"github.com/coder/serpent"
)
```

3 groups: standard library, 3rd partly libs, Coder libs.

This PR makes the change across the codebase. The PR in the stack above modifies our formatting to maintain this state of affairs, and is a separate PR so it's possible to review that one in detail.
2026-01-08 15:24:11 +04:00
Spike Curtis 49b34a716a fix: fix slog to always use array of Fields (#21426)
Upgrades to slog v3 which includes a small, but backward incompatible API change to the acceptible call arguments when logging. This change allows us to verify via compile time type checking that arguments are correct and won't cause a panic, as was possible in slog v1, which this replaces (v2 was tagged but never used in coder/coder).

It also updates dependencies that also use slog and were updated.

I've left the `aibridge` dependency as a commit SHA, under the assumption that the team there (cc @pawbana @dannykopping ) will tag and update the dependency soon and on their own schedule.

Other dependencies, I pushed new tags.
2026-01-08 10:29:41 +04:00
Steven Masley 3194bcfc9e chore: distinct operations for provisioner's 'parse', 'init', 'plan', 'apply', 'graph' (#21064)
Provisioner steps broken into smaller granular actions.
Changes:
- `ExtractArchive` moved to `init` request (was in `configure`)
- Writing `tfstate` moved to `plan` (was in `configure`)
- Moved most plan/apply outputs to `GraphComplete`
2025-12-15 11:26:41 -06:00
Steven Masley 9ca5b44b56 chore: implement persistent terraform directories (experimental) (#20563)
Prior to this, every workspace build ran `terraform init` in a fresh
directory. This would mean the `modules` are downloaded fresh. If the
module is not pinned, subsequent workspace builds would have different
modules.
2025-11-13 07:50:17 -06:00
Steven Masley 7c8deaf0d6 chore: refactor terraform paths to a central structure (#20566)
Refactors all Terraform file path logic into a centralized tfpath package. This consolidates all path construction into a single, testable Layout type. 

Instead of passing around `string` for directories, pass around the `Layout` which has the file location methods on it.
2025-11-12 09:24:07 -06:00
Steven Masley c1341cccdd feat: use proto streams to increase maximum module files payload (#18268)
This PR implements protobuf streaming to handle large module files by:
1. **Streaming large payloads**: When module files exceed the 4MB limit,
they're streamed in chunks using a new UploadFile RPC method
2. **Database storage**: Streamed files are stored in the database and
referenced by hash for deduplication
3. **Backward compatibility**: Small module files continue using the
existing direct payload method
2025-06-13 12:46:26 -05:00
Cian Johnston 7c2fb66181 chore(provisionersdk): ignore io.EOF in Session.requestReader (#18234)
My understanding is that `io.EOF` is eventually expected, so logging it
as an error may be confusing. For other errors we should definitely
WARN.

```
[info]  provisionerd-ip-172-31-12-44-14: recv done on Session  session_id=22b9ef8a-9cd6-4188-98e0-573a50d724cc  error=EOF
```
2025-06-06 09:05:22 +01:00
Marcin Tojek 0e5eecd7da feat: add more logging around echo tar (#10731) 2023-11-16 16:52:04 +01:00
Ammar Bandukwala b63dfe7b75 perf(cli): optimize CPU consumption of help pages (#9607)
This change reduces the CPU consumption of --help by ~50%.

Also, this change removes ANSI escape codes from our golden files. I
don't think those were worth the inability to parallelize golden file tests and
global state fragility.
2023-09-14 19:48:29 -05:00
Steven Masley 0e4d6896e3 test: fix cleanup order on provisioner daemon work dir (#9668)
* test: fix cleanup order on provisioner daemon work dir

* Reduce the test race condition
2023-09-13 13:13:08 -05:00
Marcin Tojek 67fe3ae8d6 feat: clean stale provisioner files (#9545) 2023-09-11 07:37:14 +00:00
Spike Curtis 60d5002eb6 refactor: change template archive extraction to be on provisioner (#9264)
* refactor provisionersdk protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisioners to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisionerd to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor tests & proto renames

* Fixes from self-review

Signed-off-by: Spike Curtis <spike@coder.com>

* appease fmt & link

Signed-off-by: Spike Curtis <spike@coder.com>

* code review fixes & e2e fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* More fmt

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* new gen; use uuid for session workdir

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert nix-based gen CI task until dogfood is on nix

Signed-off-by: Spike Curtis <spike@coder.com>

* revert deleting dogfood Docker stuff

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert "revert deleting dogfood Docker stuff"

This reverts commit 9762158167.

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-25 06:10:15 +00:00