Commit Graph

263 Commits

Author SHA1 Message Date
Cian Johnston b0ba798f78 chore(coderd/provisionerdserver): remove dangerous usage of dbtestutil.DisableForeignKeysAndTriggers (#19122)
May have caught https://github.com/coder/coder/issues/18776
2025-08-04 21:51:33 +01:00
Benjamin Peinhardt e4dc2d9418 fix: add constraint and runtime check for provisioner logs size limit (#18893)
This PR sets a constraint of 1MB on the provisioner job logs written to
the database. This is consistent with the constraint we place on
workspace agent logs:
https://github.com/coder/coder/blob/4ac6be6d835dc36c242e35a26b584b784040bf28/coderd/database/dump.sql#L2030

It also adds a message printed to the front end about the provisioner
log overflow, and updates the message printed to the front end when
workspace startup logs exceed the max, as it was causing some customers
to think their startup script had failed to run.
2025-07-30 19:09:53 -05:00
Susana Ferreira 0672bf5084 feat: support icon and description in preset (#18977)
## Description 

This PR adds support for `description` and `icon` fields to
`template_version_presets`. These fields will allow displaying richer
information for presets in the UI, improving the user experience when
creating a workspace.
Both fields are optional, non-nullable, and default to empty strings.

## Changes

* Database migration with the addition of `description VARCHAR(128)` and
`icon VARCHAR(256)` columns to the `template_version_presets` table.
* Updated the `CreateWorkspacePageView` in the UI

Note: UI changes will be addressed in a separate PR
2025-07-28 15:02:26 +01:00
Danny Kopping 0238f2926d feat: persist AI task state in template imports & workspace builds (#18449) 2025-06-24 10:36:37 +00:00
Danny Kopping 6cc4cfa346 feat: allow for default presets (#18445) 2025-06-24 12:19:19 +02:00
Danny Kopping 4699393522 fix: upsert coder_app resources in case they are persistent (#18509) 2025-06-23 18:50:44 +00:00
Danny Kopping 66e8dbbe17 feat: persist generated coder_app id (#18487) 2025-06-23 08:46:18 +00:00
ケイラ fae30a00fd chore: remove unnecessary redeclarations in for loops (#18440) 2025-06-20 13:16:55 -06:00
Yevhenii Shcherbina 0f6ca55238 feat: implement scheduling mechanism for prebuilds (#18126)
Closes https://github.com/coder/internal/issues/312
Depends on https://github.com/coder/terraform-provider-coder/pull/408

This PR adds support for defining an **autoscaling block** for
prebuilds, allowing number of desired instances to scale dynamically
based on a schedule.

Example usage:
```
data "coder_workspace_preset" "us-nix" {
  ...
  
  prebuilds = {
    instances = 0                  # default to 0 instances
    
    scheduling = {
      timezone = "UTC"             # a single timezone is used for simplicity
      
      # Scale to 3 instances during the work week
      schedule {
        cron = "* 8-18 * * 1-5"    # from 8AM–6:59PM, Mon–Fri, UTC
        instances = 3              # scale to 3 instances
      }
      
      # Scale to 1 instance on Saturdays for urgent support queries
      schedule {
        cron = "* 8-14 * * 6"      # from 8AM–2:59PM, Sat, UTC
        instances = 1              # scale to 1 instance
      }
    }
  }
}
```

### Behavior
- Multiple `schedule` blocks per `prebuilds` block are supported.
- If the current time matches any defined autoscaling schedule, the
corresponding number of instances is used.
- If no schedule matches, the **default instance count**
(`prebuilds.instances`) is used as a fallback.

### Why
This feature allows prebuild instance capacity to adapt to predictable
usage patterns, such as:
- Scaling up during business hours or high-demand periods
- Reducing capacity during off-hours to save resources

### Cron specification
The cron specification is interpreted as a **continuous time range.**

For example, the expression:

```
* 9-18 * * 1-5
```

is intended to represent a continuous range from **09:00 to 18:59**,
Monday through Friday.

However, due to minor implementation imprecision, it is currently
interpreted as a range from **08:59:00 to 18:58:59**, Monday through
Friday.

This slight discrepancy arises because the evaluation is based on
whether a specific **point in time** falls within the range, using the
`github.com/coder/coder/v2/coderd/schedule/cron` library, which performs
per-minute matching rather than strict range evaluation.

---------

Co-authored-by: Danny Kopping <danny@coder.com>
2025-06-19 11:08:48 -04:00
Jon Ayers 8b27983d14 fix: fix TestAcquireJobWithCancel_Cancel flake (#18441) 2025-06-18 22:51:13 -04:00
Jon Ayers b0fa3275d2 fix: increase TestAcquireJob_LongPoll timeout to prevent flakiness (#18442)
I'll be honest I'm not even really sure the point of this test but it
was failing due to

```
2025-06-16T15:01:54.0863251Z         	Error:      	Received unexpected error:
2025-06-16T15:01:54.0863554Z         	            	acquire job:
2025-06-16T15:01:54.0864230Z         	            	    github.com/coder/coder/v2/coderd/provisionerdserver.(*server).AcquireJob
2025-06-16T15:01:54.0865173Z         	            	        /home/runner/work/coder/coder/coderd/provisionerdserver/provisionerdserver.go:329
2025-06-16T15:01:54.0865683Z         	            	  - failed to acquire job:
2025-06-16T15:01:54.0866374Z         	            	    github.com/coder/coder/v2/coderd/provisionerdserver.(*Acquirer).AcquireJob
2025-06-16T15:01:54.0867262Z         	            	        /home/runner/work/coder/coder/coderd/provisionerdserver/acquirer.go:148
2025-06-16T15:01:54.0867819Z         	            	  - pq: canceling statement due to user request
```

which is certainly unintended.
2025-06-19 02:50:53 +00:00
Steven Masley 5a890c4aec test: fix TestUploadFileLargeModuleFiles flake, context to subtest (#18395)
Declared context outside the subtest t.Parallel.
2025-06-16 18:25:31 +00:00
Steven Masley c1341cccdd feat: use proto streams to increase maximum module files payload (#18268)
This PR implements protobuf streaming to handle large module files by:
1. **Streaming large payloads**: When module files exceed the 4MB limit,
they're streamed in chunks using a new UploadFile RPC method
2. **Database storage**: Streamed files are stored in the database and
referenced by hash for deduplication
3. **Backward compatibility**: Small module files continue using the
existing direct payload method
2025-06-13 12:46:26 -05:00
Hugo Dutka 910858b731 chore(coderd/provisionerdserver): convert dbmem tests to use postgres (#18278) 2025-06-09 10:05:29 +02:00
Steven Masley 0428c5ec1c chore: include 'everyone' group in template importing (#18257) 2025-06-05 19:25:36 +00:00
Steven Masley 8387dd27ab chore: add form_type parameter argument to db (#17920)
`form_type` is a new parameter field in the terraform provider. Bring
that field into coder/coder.

Validation for `multi-select` has also been added.
2025-05-29 08:55:19 -05:00
Cian Johnston 776c144128 fix(coderd): ensure agent timings are non-zero on insert (#18065)
Relates to https://github.com/coder/coder/issues/15432

Ensures that no workspace build timings with zero values for started_at or ended_at are inserted into the DB or returned from the API.
2025-05-29 13:36:06 +01:00
ケイラ 9fc3329575 feat: persist app groups in the database (#17977) 2025-05-27 13:13:08 -06:00
Susana Ferreira 6f6e73af03 feat: implement expiration policy logic for prebuilds (#17996)
## Summary 

This PR introduces support for expiration policies in prebuilds. The TTL
(time-to-live) is retrieved from the Terraform configuration
([terraform-provider-coder
PR](https://github.com/coder/terraform-provider-coder/pull/404)):
```
prebuilds = {
	  instances = 2
	  expiration_policy {
		  ttl = 86400
	  }
  }
```
**Note**: Since there is no need for precise TTL enforcement down to the
second, in this implementation expired prebuilds are handled in a single
reconciliation cycle: they are deleted, and new instances are created
only if needed to match the desired count.

## Changes

* The outcome of a reconciliation cycle is now expressed as a slice of
reconciliation actions, instead of a single aggregated action.
* Adjusted reconciliation logic to delete expired prebuilds and
guarantee that the number of desired instances is correct.
* Updated relevant data structures and methods to support expiration
policies parameters.
* Added documentation to `Prebuilt workspaces` page
* Update `terraform-provider-coder` to version 2.5.0:
https://github.com/coder/terraform-provider-coder/releases/tag/v2.5.0

Depends on: https://github.com/coder/terraform-provider-coder/pull/404
Fixes: https://github.com/coder/coder/issues/17916
2025-05-26 20:31:24 +01:00
Michael Suchacz b7462fb256 feat: improve transaction safety in CompleteJob function (#17970)
This PR refactors the CompleteJob function to use database transactions
more consistently for better atomicity guarantees. The large function
was broken down into three specialized handlers:

- completeTemplateImportJob
- completeWorkspaceBuildJob
- completeTemplateDryRunJob

Each handler now uses the Database.InTx wrapper to ensure all database
operations for a job completion are performed within a single
transaction, preventing partial updates in case of failures.

Added comprehensive tests for transaction behavior for each job type.

Fixes #17694

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-05-21 16:48:51 +02:00
Thomas Kosiewski 1bacd82e80 feat: add API key scope to restrict access to user data (#17692) 2025-05-15 15:32:52 +01:00
Danny Kopping f2edcf3f59 fix: add missing clause for tracking replacements (#17849)
We should only be tracking resource replacements during a prebuild
claim.

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-05-15 13:02:30 +02:00
Steven Masley 789c4beba7 chore: add dynamic parameter error if missing metadata from provisioner (#17809) 2025-05-14 12:21:36 -05:00
Danny Kopping 6e967780c9 feat: track resource replacements when claiming a prebuilt workspace (#17571)
Closes https://github.com/coder/internal/issues/369

We can't know whether a replacement (i.e. drift of terraform state
leading to a resource needing to be deleted/recreated) will take place
apriori; we can only detect it at `plan` time, because the provider
decides whether a resource must be replaced and it cannot be inferred
through static analysis of the template.

**This is likely to be the most common gotcha with using prebuilds,
since it requires a slight template modification to use prebuilds
effectively**, so let's head this off before it's an issue for
customers.

Drift details will now be logged in the workspace build logs:


![image](https://github.com/user-attachments/assets/da1988b6-2cbe-4a79-a3c5-ea29891f3d6f)

Plus a notification will be sent to template admins when this situation
arises:


![image](https://github.com/user-attachments/assets/39d555b1-a262-4a3e-b529-03b9f23bf66a)

A new metric - `coderd_prebuilt_workspaces_resource_replacements_total`
- will also increment each time a workspace encounters replacements.

We only track _that_ a resource replacement occurred, not how many. Just
one is enough to ruin a prebuild, but we can't know apriori which
replacement would cause this.
For example, say we have 2 replacements: a `docker_container` and a
`null_resource`; we don't know which one might
cause an issue (or indeed if either would), so we just track the
replacement.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-05-14 14:52:22 +02:00
Sas Swart 425ee6fa55 feat: reinitialize agents when a prebuilt workspace is claimed (#17475)
This pull request allows coder workspace agents to be reinitialized when
a prebuilt workspace is claimed by a user. This facilitates the transfer
of ownership between the anonymous prebuilds system user and the new
owner of the workspace.

Only a single agent per prebuilt workspace is supported for now, but
plumbing has already been done to facilitate the seamless transition to
multi-agent support.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: Danny Kopping <dannykopping@gmail.com>
2025-05-14 14:15:36 +02:00
Danielle Maywood 0b5f27f566 feat: add parent_id column to workspace_agents table (#17758)
Adds a new nullable column `parent_id` to `workspace_agents` table. This
lays the groundwork for having child agents.
2025-05-13 00:01:31 +01:00
Steven Masley 398b999d8f chore: pass previous values into terraform apply (#17696)
Pass previous workspace build parameter values into the terraform
`plan/apply`. Enforces monotonicity in terraform as well as `coderd`.
2025-05-12 15:32:00 -05:00
ケイラ d0ab91c16f fix: reduce size of terraform modules archive (#17749) 2025-05-12 13:50:07 -06:00
Steven Masley 37832413ba chore: resolve internal drpc package conflict (#17770)
Our internal drpc package name conflicts with the external one in usage. 
`drpc.*` == external
`drpcsdk.*` == internal
2025-05-12 10:31:38 -05:00
Danny Kopping af2941bb92 feat: add is_prebuild_claim to distinguish post-claim provisioning (#17757)
Used in combination with
https://github.com/coder/terraform-provider-coder/pull/396

This is required by both https://github.com/coder/coder/pull/17475 and
https://github.com/coder/coder/pull/17571

Operators may need to conditionalize their templates to perform certain
operations once a prebuilt workspace has been claimed. This value will
**only** be set once a claim takes place and a subsequent `terraform
apply` occurs. Any `terraform apply` runs thereafter will be
indistinguishable from a normal run on a workspace.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-05-12 14:19:03 +00:00
Jon Ayers a9f1a6b2a2 fix: revert fix: persist terraform modules during template import (#17665) (#17734)
This reverts commit ae3d90b057.
2025-05-08 22:03:08 -04:00
ケイラ ae3d90b057 fix: persist terraform modules during template import (#17665) 2025-05-08 16:13:46 -06:00
Yevhenii Shcherbina 118f12ac3a feat: implement claiming of prebuilt workspaces (#17458)
Signed-off-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: Danny Kopping <danny@coder.com>
Co-authored-by: Edward Angert <EdwardAngert@users.noreply.github.com>
Co-authored-by: EdwardAngert <17991901+EdwardAngert@users.noreply.github.com>
Co-authored-by: Jaayden Halko <jaayden.halko@gmail.com>
Co-authored-by: Ethan <39577870+ethanndickson@users.noreply.github.com>
Co-authored-by: M Atif Ali <atif@coder.com>
Co-authored-by: Aericio <16523741+Aericio@users.noreply.github.com>
Co-authored-by: M Atif Ali <me@matifali.dev>
Co-authored-by: Michael Suchacz <203725896+ibetitsmike@users.noreply.github.com>
2025-04-24 09:39:38 -04:00
Steven Masley 71dbd0c888 fix: nil ptr deref when removing OIDC from deployment and accessing old users (#17501)
If OIDC is removed from a deployment, trying to create a workspace for a previous user
on OIDC would panic.
2025-04-23 08:45:26 -05:00
Steven Masley ea65ddc17d fix: correct user roles being passed into terraform context (#17460)
Roles were being passed into the workspace context incorrectly. Site
wide scopes were being org scoped. Roles outside the org should also not
be sent.
2025-04-17 15:42:23 -05:00
ケイラ f670bc31f5 chore: update testutil chan helpers (#17408) 2025-04-16 10:37:09 -06:00
ケイラ 2a76f5028e fix: don't attempt to insert empty terraform plans into the database (#17426) 2025-04-16 10:14:35 -06:00
Sas Swart a98605913a feat: mark prebuilds as such and set their preset ids (#16965)
This pull request closes https://github.com/coder/internal/issues/513
2025-04-14 15:34:50 +02:00
Sas Swart 0b2b643ce2 feat: persist prebuild definitions on template import (#16951)
This PR allows provisioners to recognise and report prebuild definitions
to the coder control plane. It also allows the coder control plane to
then persist these to its store.

closes https://github.com/coder/internal/issues/507

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: evgeniy-scherbina <evgeniy.shcherbina.es@gmail.com>
2025-04-07 10:35:28 +02:00
Sas Swart 99c6f235eb feat: add migrations and queries to support prebuilds (#16891)
Depends on https://github.com/coder/coder/pull/16916 _(change base to
`main` once it is merged)_

Closes https://github.com/coder/internal/issues/514

_This is one of several PRs to decompose the `dk/prebuilds` feature
branch into separate PRs to merge into `main`._

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: evgeniy-scherbina <evgeniy.shcherbina.es@gmail.com>
2025-04-03 10:58:30 +02:00
Edward Angert 0ec87abaa5 docs: add new section on managing provisioners from the dashboard (#16563)
closes #16513 


[preview](https://coder.com/docs/@16513-manage-ext-provisioners/admin/provisioners/manage-provisioner-jobs)

---------

Co-authored-by: EdwardAngert <17991901+EdwardAngert@users.noreply.github.com>
2025-04-01 21:04:31 -05:00
Mathias Fredriksson 7d4b3c8634 feat(agent): add devcontainer autostart support (#17076)
This change adds support for devcontainer autostart in workspaces. The
preconditions for utilizing this feature are:

1. The `coder_devcontainer` resource must be defined in Terraform
2. By the time the startup scripts have completed,
	- The `@devcontainers/cli` tool must be installed
	- The given workspace folder must contain a devcontainer configuration

Example Terraform:

```tf
resource "coder_devcontainer" "coder" {
  agent_id         = coder_agent.main.id
  workspace_folder = "/home/coder/coder"
  config_path      = ".devcontainer/devcontainer.json" # (optional)
}
```

Closes #16423
2025-03-27 12:31:30 +02:00
Jon Ayers 17ddee05e5 chore: update golang to 1.24.1 (#17035)
- Update go.mod to use Go 1.24.1
- Update GitHub Actions setup-go action to use Go 1.24.1
- Fix linting issues with golangci-lint by:
  - Updating to golangci-lint v1.57.1 (more compatible with Go 1.24.1)

🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <claude@anthropic.com>
2025-03-26 01:56:39 -05:00
Mathias Fredriksson 5c8cac9fb7 feat: add name to workspace agent devcontainers (#17089)
In the presence of multiple devcontainers, it would be nice to
differentiate them by name. This change inherits the resource name from
terraform.

Refs #17076
2025-03-25 12:59:20 +00:00
ケイラ 5b3eda6719 chore: persist template import terraform plan in postgres (#17012) 2025-03-24 10:01:50 -06:00
Mathias Fredriksson 69ba27e347 feat: allow specifying devcontainer on agent in terraform (#16997)
This change allows specifying devcontainers in terraform and plumbs it
through to the agent via agent manifest.

This will be used for autostarting devcontainers in a workspace.

Depends on coder/terraform-provider-coder#368
Updates #16423
2025-03-20 19:09:39 +02:00
Eng Zer Jun 04c33968cf refactor: replace golang.org/x/exp/slices with slices (#16772)
The experimental functions in `golang.org/x/exp/slices` are now
available in the standard library since Go 1.21.

Reference: https://go.dev/doc/go1.21#slices

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-03-04 00:46:49 +11:00
Nick Fisher ca23abe12c feat(provisioner): add support for workspace_owner_rbac_roles (#16407)
Part of https://github.com/coder/terraform-provider-coder/pull/330

Adds support for the coder_workspace_owner.rbac_roles attribute
2025-03-02 14:54:44 -06:00
Dean Sheather 9469b78290 fix!: enforce regex for agent names (#16641)
Underscores and double hyphens are now blocked. The regex is almost the
exact same as the `coder_app` `slug` regex, but uppercase characters are
still permitted.
2025-02-20 05:09:26 +00:00
Ethan 3fddfef879 fix!: enforce agent names be case-insensitive-unique per-workspace (#16614)
Relates to https://github.com/coder/coder-desktop-macos/issues/54

Currently, it's possible to have two agents within the same workspace whose names only differ in capitalization:
This leads to an ambiguity in two cases:
- For CoderVPN, we'd like to allow support to workspaces with a hostname of the form: `agent.workspace.username.coder`.
- Workspace apps (`coder_app`s) currently use subdomains of the form: `<app>--<agent>--<workspace>--<username>(--<suffix>)?`.

Of note is that DNS hosts must be strictly lower case, hence the ambiguity.

This fix is technically a breaking change, but only for the incredibly rare use case where a user has:
- A workspace with two agents
- Those agent names differ only in capitalization.

Those templates & workspaces will now fail to build. This can be fixed by choosing wholly unique names for the agents.
2025-02-20 12:51:25 +11:00