Commit Graph

44 Commits

Author SHA1 Message Date
Asher 4d414a0df7 feat: add --use-parameter-defaults flag (#21119)
This is like `--yes`, but for parameter prompts.
2026-01-16 17:04:57 -09:00
Steven Masley 60b3fd0783 chore!: send modules archive over the proto messages (#21398)
# What this does

Dynamic parameters caches the `./terraform/modules` directory for parameter usage. What this PR does is send over this archive to the provisioner when building workspaces.

This allow terraform to skip downloading modules from their registries, a step that takes seconds. 

<img width="1223" height="429" alt="Screenshot From 2025-12-29 12-57-52" src="https://github.com/user-attachments/assets/16066e0a-ac79-4296-819d-924f4b0418dc" />


# Wire protocol

The wire protocol reuses the same mechanism used to download the modules `provisoner -> coder`. It splits up large archives into multiple protobuf messages so larger archives can be sent under the message size limit.

# 🚨  Behavior Change (Breaking Change) 🚨 

**Before this PR** modules were downloaded on every workspace build. This means unpinned modules always fetched the latest version

**After this PR** modules are cached at template import time, and their versions are effectively pinned for all subsequent workspace builds.
2026-01-09 11:33:34 -06:00
Spike Curtis bddb808b25 chore: arrange imports in a standard way (#21452)
Fixes all our Go file imports to match the preferred spec that we've _mostly_ been using. For example:

```
import (
	"context"
	"time"

	"github.com/prometheus/client_golang/prometheus"
	"golang.org/x/xerrors"
	"gopkg.in/natefinch/lumberjack.v2"

	"cdr.dev/slog/v3"
	"github.com/coder/coder/v2/codersdk/agentsdk"
	"github.com/coder/serpent"
)
```

3 groups: standard library, 3rd partly libs, Coder libs.

This PR makes the change across the codebase. The PR in the stack above modifies our formatting to maintain this state of affairs, and is a separate PR so it's possible to review that one in detail.
2026-01-08 15:24:11 +04:00
Spike Curtis 49b34a716a fix: fix slog to always use array of Fields (#21426)
Upgrades to slog v3 which includes a small, but backward incompatible API change to the acceptible call arguments when logging. This change allows us to verify via compile time type checking that arguments are correct and won't cause a panic, as was possible in slog v1, which this replaces (v2 was tagged but never used in coder/coder).

It also updates dependencies that also use slog and were updated.

I've left the `aibridge` dependency as a commit SHA, under the assumption that the team there (cc @pawbana @dannykopping ) will tag and update the dependency soon and on their own schedule.

Other dependencies, I pushed new tags.
2026-01-08 10:29:41 +04:00
Steven Masley 3194bcfc9e chore: distinct operations for provisioner's 'parse', 'init', 'plan', 'apply', 'graph' (#21064)
Provisioner steps broken into smaller granular actions.
Changes:
- `ExtractArchive` moved to `init` request (was in `configure`)
- Writing `tfstate` moved to `plan` (was in `configure`)
- Moved most plan/apply outputs to `GraphComplete`
2025-12-15 11:26:41 -06:00
Steven Masley 7c8deaf0d6 chore: refactor terraform paths to a central structure (#20566)
Refactors all Terraform file path logic into a centralized tfpath package. This consolidates all path construction into a single, testable Layout type. 

Instead of passing around `string` for directories, pass around the `Layout` which has the file location methods on it.
2025-11-12 09:24:07 -06:00
Spike Curtis 289f0217c7 feat: add scaletest Runner for dynamicparameters load gen (#19890)
relates to https://github.com/coder/internal/issues/912

Adds a new scaletest Runner to generate dynamic parameters load.

A later PR will add the CLI command, including creating the template & version.
2025-09-25 16:18:37 +04:00
Jaayden Halko 1320b8d5be feat: make dynamic parameters opt-in by default for new templates (#19006)
resolves #18975 

---------

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
Co-authored-by: blink-so[bot] <211532188+blink-so[bot]@users.noreply.github.com>
2025-07-28 20:41:49 -05:00
Thomas Kosiewski 1bacd82e80 feat: add API key scope to restrict access to user data (#17692) 2025-05-15 15:32:52 +01:00
ケイラ d0ab91c16f fix: reduce size of terraform modules archive (#17749) 2025-05-12 13:50:07 -06:00
Steven Masley 37832413ba chore: resolve internal drpc package conflict (#17770)
Our internal drpc package name conflicts with the external one in usage. 
`drpc.*` == external
`drpcsdk.*` == internal
2025-05-12 10:31:38 -05:00
Jon Ayers a9f1a6b2a2 fix: revert fix: persist terraform modules during template import (#17665) (#17734)
This reverts commit ae3d90b057.
2025-05-08 22:03:08 -04:00
ケイラ ae3d90b057 fix: persist terraform modules during template import (#17665) 2025-05-08 16:13:46 -06:00
ケイラ 859dd2fc3f feat: add dynamic parameters websocket endpoint (#17165) 2025-04-10 14:08:50 -06:00
ケイラ ca414b031a fix: fix data race in echo provisioner (#17142) 2025-03-27 18:04:05 -06:00
ケイラ 5b3eda6719 chore: persist template import terraform plan in postgres (#17012) 2025-03-24 10:01:50 -06:00
Marcin Tojek 13cbca679e feat: support template bundles as zip archives (#11839) 2024-01-31 14:49:55 +01:00
Spike Curtis 9a4e1100fa chore: move drpc transport tools to codersdk/drpc (#11224)
Part of #10532

DRPC transport over yamux and in-mem pipes was previously only used on the provisioner APIs, but now will also be used in tailnet.  Moved to subpackage of codersdk to avoid import loops.
2023-12-15 12:41:39 +04:00
Marcin Tojek 0e5eecd7da feat: add more logging around echo tar (#10731) 2023-11-16 16:52:04 +01:00
Kyle Carberry 5596fb20b5 chore: move /gitauth to /externalauth on the frontend (#9954)
* chore: move `/gitauth` to `/externalauth` on the frontend

This actually took a lot more jank than anticipated,
so I wanted to split this up before adding the ability
to embed new providers.

* Rename FE

* Fix em' up

* Fix linting error

* Fix e2e tests

* chore: update helm golden files
2023-09-30 14:30:01 -05:00
Spike Curtis 60d5002eb6 refactor: change template archive extraction to be on provisioner (#9264)
* refactor provisionersdk protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisioners to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisionerd to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor tests & proto renames

* Fixes from self-review

Signed-off-by: Spike Curtis <spike@coder.com>

* appease fmt & link

Signed-off-by: Spike Curtis <spike@coder.com>

* code review fixes & e2e fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* More fmt

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* new gen; use uuid for session workdir

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert nix-based gen CI task until dogfood is on nix

Signed-off-by: Spike Curtis <spike@coder.com>

* revert deleting dogfood Docker stuff

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert "revert deleting dogfood Docker stuff"

This reverts commit 9762158167.

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-25 06:10:15 +00:00
Ammar Bandukwala 545a256b57 fix: correctly reject quota-violating builds (#9233)
Due to a logical error in CommitQuota, all workspace Stop->Start operations
were being accepted, regardless of the Quota limit. This issue only
appeared after #9201, so this was a minor regression in main for about
3 days. This PR adds a test to make sure this kind of bug doesn't recur.

To make the new test possible, we give the echo provisioner the ability
to simulate responses to specific transitions.
2023-08-22 02:55:39 +00:00
Kyle Carberry 22e781eced chore: add /v2 to import module path (#9072)
* chore: add /v2 to import module path

go mod requires semantic versioning with versions greater than 1.x

This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```

Migrate generated files to import /v2

* Fix gen
2023-08-18 18:55:43 +00:00
Jon Ayers 1b0124ecdb feat: automatically stop workspaces based on failure_ttl (#7989) 2023-06-22 00:33:22 -04:00
Marcin Tojek a7366a8b76 feat!: drop support for legacy parameters (#7663) 2023-06-02 11:16:46 +02:00
Marcin Tojek 0ba200c2a1 feat: Enable workspace debug logging (#6838)
* feat: Enable workspace debug logging

* Fix

* Fix

* Fix

* fix

* fix

* Enable RBAC

* unit tests

* Fix

* fix

* fix

* fix

* more tests

* fix: workspacebuild_test use roles

* fix: swagger comment

* fix: ctx.Done

* fix: address PR comments

* break loop
2023-03-30 16:00:33 +02:00
Kyle Carberry aaa3b31a0b chore: add echo helper to create an agent with token (#6576)
This should reduce some LOC and duplication in tests!
2023-03-21 18:03:38 +00:00
Ammar Bandukwala 7a52a9cfc8 fix: flake in template pull (#6317) 2023-02-23 04:59:07 +00:00
Ammar Bandukwala f7c10adb04 feat(cli): extract tar in template pull (#6289) 2023-02-22 19:29:51 +00:00
Ammar Bandukwala f05609b4da chore: format Go more aggressively 2023-02-18 18:32:09 -06:00
Ammar Bandukwala 592ce3b118 feat(cli): allow direct tar upload in template update/create (#5720) 2023-01-16 14:32:11 -06:00
Colin Adler 1f20cab110 fix: don't use yamux for in-memory provisioner{,d} streams (#5136) 2022-11-22 12:19:32 -06:00
Ammar Bandukwala 95fb59696e Refactor Provisioner to distinguish Plan and Apply (#5036) 2022-11-11 16:45:58 -06:00
Jon Ayers 2353687610 feat: unexpose coderdtest.NewWithAPI (#2613)
* feat: unexpose coderdtest.NewWithAPI
2022-06-27 13:50:52 -05:00
Kyle Carberry 33a04f661f fix: Check if start is nil before consuming in echo provisioner (#2686)
This caused a race seen here:
https://github.com/coder/coder/runs/7074123929?check_suite_focus=true#step:10:217
2022-06-27 09:57:37 -05:00
Kyle Carberry 024ab6df57 fix: Use in-memory filesystem for echo provisioner tests (#2408)
* fix: Use in-memory filesystem for echo provisioner tests

This should reduce IO in CI to shave some time off tests!

* test: Increase timeouts to reduce flakes

It's difficult to understand what's timing out due to a lock
vs. taking a long time. This should help resolve! 🕵️
2022-06-16 15:09:22 +00:00
Dean Sheather 6be8a373e0 feat: run a terraform plan before creating workspaces with the given template parameters (#1732) 2022-06-02 00:44:53 +10:00
Cian Johnston c2f74f3cc2 chore: avoid concurrent usage of t.FailNow (#1683)
* chore: golangci: add linter rule to report usage of t.FailNow inside goroutines
* chore: avoid t.FailNow in goroutines to appease the race detector
2022-05-24 08:58:39 +01:00
Colin Adler 98ccd0eb89 feat: add README parsing to template versions (#1500) 2022-05-17 15:00:48 -05:00
Kyle Carberry c451f4e685 feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00
Kyle Carberry bf0ae8f573 feat: Refactor API routes to use UUIDs instead of friendly names (#401)
* Add client for agent

* Cleanup code

* Fix linting error

* Rename routes to be simpler

* Rename workspace history to workspace build

* Refactor HTTP middlewares to use UUIDs

* Cleanup routes

* Compiles!

* Fix files and organizations

* Fix querying

* Fix agent lock

* Cleanup database abstraction

* Add parameters

* Fix linting errors

* Fix log race

* Lock on close wait

* Fix log cleanup

* Fix e2e tests

* Fix upstream version of opencensus-go

* Update coderdtest.go

* Fix coverpkg

* Fix codecov ignore
2022-03-07 11:40:54 -06:00
Kyle Carberry 9d2803e07a feat: Add graceful exits to provisionerd (#372)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* fix: Convert all jobs to use a common resource and agent type

This enables a consistent API for project import and provisioned resources.

* Add "coder_workspace" data source

* feat: Remove magical parameters from being injected

This is a much cleaner abstraction. Explicitly declaring the user
parameters for each provisioner makes for significantly simpler
testing.

* feat: Add graceful exits to provisionerd

Terraform (or other provisioners) may need to cleanup state, or
cancel actions before exit. This adds the ability to gracefully
exit provisionerd.

* Fix cancel error check
2022-02-28 18:40:49 +00:00
Kyle Carberry 1796dc6c2f chore: Add test helpers to improve coverage (#166)
* chore: Rename ProjectHistory to ProjectVersion

Version more accurately represents version storage. This
forks from the WorkspaceHistory name, but I think it's
easier to understand Workspace history.

* Rename files

* Standardize tests a bit more

* Remove Server struct from coderdtest

* Improve test coverage for workspace history

* Fix linting errors

* Fix coderd test leak

* Fix coderd test leak

* Improve workspace history logs

* Standardize test structure for codersdk

* Fix linting errors

* Fix WebSocket compression

* Update coderd/workspaces.go

Co-authored-by: Bryan <bryan@coder.com>

* Add test for listing project parameters

* Cache npm dependencies with setup node

* Remove windows npm cache key

Co-authored-by: Bryan <bryan@coder.com>
2022-02-05 18:24:51 -06:00
Kyle Carberry 682238d384 feat: Add echo provisioner (#162)
This replaces the cdr-basic provisioner type with
"echo". It reads binary data from the directory
and returns the responses in order.

This is used to test project and workspace job logic.
2022-02-04 16:51:54 -06:00