Commit Graph

124 Commits

Author SHA1 Message Date
Spike Curtis 211718f95a fix: fix MaliciousTar test case (#10158)
fixes #9895

Problem was that provisionerd tries to acquire the next job, and races with shutdown, triggering the assert in the handler.  Switches this test case to use the more robust handler.
2023-10-10 13:24:43 +04:00
Kyle Carberry 5596fb20b5 chore: move /gitauth to /externalauth on the frontend (#9954)
* chore: move `/gitauth` to `/externalauth` on the frontend

This actually took a lot more jank than anticipated,
so I wanted to split this up before adding the ability
to embed new providers.

* Rename FE

* Fix em' up

* Fix linting error

* Fix e2e tests

* chore: update helm golden files
2023-09-30 14:30:01 -05:00
Kyle Carberry 8abca9bea7 chore: rename git_auth to external_auth in our schema (#9935)
* chore: rename `git_auth` to `external_auth` in our schema

We're changing Git auth to be external auth. It will support
any OAuth2 or OIDC provider.

To split up the larger change I want to contribute the schema
changes first, and I'll add the feature itself in another PR.

* Fix names

* Fix outdated view

* Rename some additional places

* Fix sort order

* Fix template versions auth route

* Fix types

* Fix dbauthz
2023-09-29 19:13:20 +00:00
Spike Curtis 375c70d141 feat: integrate Acquirer for provisioner jobs (#9717)
* chore: add Acquirer to provisionerdserver pkg

Signed-off-by: Spike Curtis <spike@coder.com>

* code review improvements & fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* feat: integrate Acquirer for provisioner jobs

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix imports, whitespace

Signed-off-by: Spike Curtis <spike@coder.com>

* provisionerdserver always closes; remove poll interval from playwright

Signed-off-by: Spike Curtis <spike@coder.com>

* post jobs outside transactions

Signed-off-by: Spike Curtis <spike@coder.com>

* graceful shutdown in test

Signed-off-by: Spike Curtis <spike@coder.com>

* Mark AcquireJob deprecated

Signed-off-by: Spike Curtis <spike@coder.com>

* Graceful shutdown on all provisionerd tests

Signed-off-by: Spike Curtis <spike@coder.com>

* Deprecate, not remove CLI flags

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-09-19 10:25:57 +04:00
Jon Ayers 622442203d chore: fix test flake in TestProvisionerd (#9709) 2023-09-18 11:23:22 -05:00
Spike Curtis 11b6068112 feat: add support for networked provisioners (#9593)
* Refactor provisionerd to use interface to connect to provisioners

Signed-off-by: Spike Curtis <spike@coder.com>

* feat: add support for networked provisioners

Signed-off-by: Spike Curtis <spike@coder.com>

* fix token length and linting

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-09-08 09:53:48 +00:00
Mathias Fredriksson b240799f47 refactor(cli): use codersdk for provisioner types (#9508)
This change removes one use of `coderd/database` from the slim binary
and more correctly uses codersdk instead of database or provisionerd
packages.

No size change (yet).

Ref: #9380
2023-09-04 21:42:15 +03:00
Spike Curtis 90acf998bf fix: fix null pointer on external provisioner daemons with daily_cost (#9401)
* fix: fix null pointer on external provisioner daemons with daily_cost

Signed-off-by: Spike Curtis <spike@coder.com>

* Add logging for debounce and job acquire

Signed-off-by: Spike Curtis <spike@coder.com>

* Return error instead of panic

Signed-off-by: Spike Curtis <spike@coder.com>

* remove debounce on external provisioners to fix test flakes

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-30 10:48:35 +00:00
Spike Curtis 60d5002eb6 refactor: change template archive extraction to be on provisioner (#9264)
* refactor provisionersdk protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisioners to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisionerd to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor tests & proto renames

* Fixes from self-review

Signed-off-by: Spike Curtis <spike@coder.com>

* appease fmt & link

Signed-off-by: Spike Curtis <spike@coder.com>

* code review fixes & e2e fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* More fmt

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* new gen; use uuid for session workdir

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert nix-based gen CI task until dogfood is on nix

Signed-off-by: Spike Curtis <spike@coder.com>

* revert deleting dogfood Docker stuff

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert "revert deleting dogfood Docker stuff"

This reverts commit 9762158167.

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-25 06:10:15 +00:00
Jon Ayers 7ddb216d87 chore: revert nix-related CI changes (#9321)
* chore: revert nix-related CI changes

- Reverts using nix to run CI-dependencies.
- Running 'make gen' in a dogfood workspace resulted in inconsistent
  results for protobuf-related files making it difficult to pass CI.
  This PR imports the minimum changes necessary to make CI compatible
  with dogfood.
2023-08-24 21:26:30 -05:00
Ammar Bandukwala 545a256b57 fix: correctly reject quota-violating builds (#9233)
Due to a logical error in CommitQuota, all workspace Stop->Start operations
were being accepted, regardless of the Quota limit. This issue only
appeared after #9201, so this was a minor regression in main for about
3 days. This PR adds a test to make sure this kind of bug doesn't recur.

To make the new test possible, we give the echo provisioner the ability
to simulate responses to specific transitions.
2023-08-22 02:55:39 +00:00
Ammar Bandukwala 6b8102cf4c feat(cli): add daily_cost to coder ls (#9200) 2023-08-19 12:56:08 -05:00
Kyle Carberry 22e781eced chore: add /v2 to import module path (#9072)
* chore: add /v2 to import module path

go mod requires semantic versioning with versions greater than 1.x

This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```

Migrate generated files to import /v2

* Fix gen
2023-08-18 18:55:43 +00:00
Kyle Carberry c0a78533bf chore: reduce build time by converting docker image to build with nix (#9099)
* Add Docker image

* Try building the container in CI

* Uncomment testing lines

* Trim image step

* Hit 'em with a Docker load

* Oopsie!

* Add a tag and push!

* Fix image name 🤦

* Fix sudo

* Fix target name

* Build and push

* Fix login to docker hub

* Revert to Docker push

* Fix PATH

* Fix SHA

* Fix ca certs

* Fix ca certs

* Fix coping files in

* Fix docker init

* Fix Docker group and init

* Add comments to our Nix

* Fix build stage

* Add some more comments

* Remove old dogfood image files

* Tag and push with branch name

* Fix tag passing

* Fix tag passing

* Remove old pull triggers

* Convert gen to use Nix

* Add protobuf to the flake

* Add prettier to the dev shell

* Swap to the faster Nix cache action

* Add the correct yq

* Fix gen

* Add make to the flake

* Update extensions
2023-08-17 21:19:36 -05:00
Ammar Bandukwala be40dc85ab chore: cleanup extraneous logging (#9156)
* The batchstats warning went out on every Ctrl+C in my development

Rule of silence:

The provisioner and connect messages messages were sent out on every startup
without a corresponding user event, making them annoying and more-so
debug messages.
2023-08-17 21:01:55 +00:00
Spike Curtis 21af020386 feat: add external provisioner daemon helm chart (#8939)
* Refactor helm to extract common templates to libcoder

Signed-off-by: Spike Curtis <spike@coder.com>

* Remove comment from libcoder Chart.yaml

Signed-off-by: Spike Curtis <spike@coder.com>

* Add provisioner helm chart

* Fix prettier, linting, docs

Signed-off-by: Spike Curtis <spike@coder.com>

* Log at INFO when provisionerd connects to coderd

Signed-off-by: Spike Curtis <spike@coder.com>

* remove unnecessary exports in helm tests

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-10 13:59:43 +04:00
Asher 7ed17b2605 fix: add some missing workspace updates (#7790)
* Standardize on function to get workspace channel name

There were two, now there is one.

* Add some missing workspace updates

There are some failure cases where we do not set the type as a workspace
build which causes the workspace update to never be published.

* Make build failures warnings

Otherwise the associated test fails due to the logger fataling on
error messages.
2023-07-14 15:07:48 -08:00
Marcin Tojek c6fcd7ee93 fix: report failed CompletedJob (#8318) 2023-07-06 07:26:33 +00:00
Dean Sheather 98a5ae7f48 feat: add provisioner job hang detector (#7927) 2023-06-25 13:17:00 +00:00
Marcin Tojek 4fb4c9b270 chore: add more rules to ensure logs consistency (#8104) 2023-06-21 12:00:38 +02:00
Colin Adler 7d43f4ceee chore: update dogfood image (#8114) 2023-06-20 15:08:15 -05:00
Marcin Tojek b1d1b63113 chore: ensure logs consistency across Coder (#8083) 2023-06-20 12:30:45 +02:00
Mathias Fredriksson c12c9f1f4e chore(go.mod): update cdr.dev/slog (#7994)
* chore(mod): update cdr.dev/slog

* fix: change uses of []slog.Field to []any to match new API
2023-06-13 18:17:04 +00:00
goodspark 0665a6c2f2 feat: add metric for provisioner daemons (#7858) 2023-06-06 16:50:11 -05:00
Marcin Tojek a7366a8b76 feat!: drop support for legacy parameters (#7663) 2023-06-02 11:16:46 +02:00
Marcin Tojek 4eb0baa849 chore: update protoc-gen-go (#7708) 2023-05-30 10:24:34 +02:00
Colin Adler 8bd9f9c351 feat: unified tracing between coderd<->provisionerd (#7370) 2023-05-03 23:02:35 +00:00
Colin Adler 085330ad96 fix(provisionerd): only heartbeat when logs aren't being flushed (#7110) 2023-04-13 14:02:10 -05:00
ElliotG 0069831e8d fix: use error log when failing provisioner job (#6812)
Co-authored-by: Colin Adler <colin1adler@gmail.com>
2023-04-05 13:30:53 -05:00
Colin Adler a29fc7dd6f chore: update otel to v1.14.0 (#6963) 2023-04-03 00:31:39 -05:00
Marcin Tojek 0ba200c2a1 feat: Enable workspace debug logging (#6838)
* feat: Enable workspace debug logging

* Fix

* Fix

* Fix

* fix

* fix

* Enable RBAC

* unit tests

* Fix

* fix

* fix

* fix

* more tests

* fix: workspacebuild_test use roles

* fix: swagger comment

* fix: ctx.Done

* fix: address PR comments

* break loop
2023-03-30 16:00:33 +02:00
Kyle Carberry df31636e72 feat: pass access_token to coder_git_auth resource (#6713)
This allows template authors to leverage git auth to perform
custom actions, like clone repositories.
2023-03-22 19:37:08 +00:00
Marcin Tojek 7587850a1c feat: import value from legacy variable to build parameter (#6556) 2023-03-14 12:02:44 +00:00
Marcin Tojek 3b87316ad7 feat: propagate job error codes (#6507)
* feat: propagate job error_code

* fix

* Fix

* Fix

* Fix

* add errors to typesGenerated

* Address PR comments

* Fix
2023-03-08 16:32:00 +01:00
Cian Johnston 26a725f86d fix: provisionerd: add more context to logs emitted, fix log level (#6508)
- Previously, we were logging all provision response logs at level INFO, regardless of the log level of the log streamed from the provisioner. We now log these at the original level (defaulting to INFO).
- Now logging "provision failed" message at level ERROR WARN and including the error field in the message.
2023-03-08 15:12:52 +00:00
Cian Johnston 1483b42259 fix(provisionerd): emit workspace build transition in lower case (#6455) 2023-03-06 11:14:19 +00:00
Kyle Carberry 7f226d4f90 feat: add support for coder_git_auth data source (#6334)
* Add git auth providers schema

* Pipe git auth providers to the schema

* Add git auth providers to the API

* Add gitauth endpoint to query authenticated state

* Add endpoint to query git state

* Use BroadcastChannel to automatically authenticate with Git

* Add error validation for submitting the create workspace form

* Fix panic on template dry-run

* Add tests for the template version Git auth endpoint

* Show error if no gitauth is configured

* Add gitauth to cliui

* Fix unused method receiver

* Fix linting errors

* Fix dbauthz querier test

* Fix make gen

* Add JavaScript test for git auth

* Fix bad error message

* Fix provisionerd test race

See https://github.com/coder/coder/actions/runs/4277960646/jobs/7447232814

* Fix requested changes

* Add comment to CreateWorkspacePageView
2023-02-27 10:18:19 -06:00
Cian Johnston 473ab208af fix: replace owner_email with workspace_owner, change action to workspace_transition (#6337) 2023-02-24 04:38:58 +00:00
Cian Johnston 282507f0fb fix(provisionerd): add more detailed workspace build logging (#6331) 2023-02-24 00:03:12 +00:00
Marcin Tojek 50db90c33d fix: rephrase rich parameters error (#6327) 2023-02-23 21:49:40 +00:00
Cian Johnston 43e8ba0811 feat(api): add prometheus metric coderd_workspace_builds_total (#6314)
This PR adds the prometheus metric coderd_workspace_builds_total.
It measures the total number of workspace builds, along with a number of labels intended to be useful for an operator debugging a failed workspace build trying to discover the scope of the issue.
2023-02-23 01:28:10 +00:00
Ammar Bandukwala f05609b4da chore: format Go more aggressively 2023-02-18 18:32:09 -06:00
Marcin Tojek 3b7b96ac28 feat: Expose managed variables via API (#6134)
* WIP

* hcl

* useManagedVariables

* fix

* Fix

* Fix

* fix

* go:build

* Fix

* fix: bool flag

* Insert template variables

* API

* fix

* Expose via API

* More wiring

* CLI for testing purposes

* WIP

* Delete FIXME

* planVars

* WIP

* WIP

* UserVariableValues

* no dry run

* Dry run

* Done FIXME

* Fix

* Fix: CLI

* Fix: migration

* API tests

* Test info

* Tests

* More tests

* fix: lint

* Fix: authz

* Address PR comments

* Fix

* fix

* fix
2023-02-15 18:24:15 +01:00
Cian Johnston f0f39b4892 chore: break down dbauthz.System into smaller roles (#6218)
- rbac: export rbac.Permissions
- dbauthz: move GetDeploymentDAUs, GetTemplateDAUs,
  GetTemplateAverageBuildTime from querier.go to system.go
  and removes auth checks
- dbauthz: remove AsSystem(), add individual roles for
  autostart, provisionerd, add restricted system role for 
  everything else
2023-02-15 16:14:37 +00:00
Mathias Fredriksson 860e2829c5 fix: Prevent race between provisionerd connect and close (#6206)
* fix: Prevent race between provisionerd connect and close

* test: Add detection for provisioner creation after test completion
2023-02-14 16:37:43 +00:00
Mathias Fredriksson 41ae01d2e9 fix: Improve closure of provisioner and agent tailnet dial (#6199) 2023-02-14 14:57:48 +00:00
Steven Masley 6fb8aff6d0 feat: Add initial AuthzQuerier implementation (#5919)
feat: Add initial AuthzQuerier implementation
- Adds package database/dbauthz that adds a database.Store implementation where each method goes through AuthZ checks
- Implements all database.Store methods on AuthzQuerier
- Updates and fixes unit tests where required
- Updates coderd initialization to use AuthzQuerier if codersdk.ExperimentAuthzQuerier is enabled
2023-02-14 14:27:06 +00:00
Kyle Carberry 2d0a69ba47 fix: require client pipe to be closed in provisionerd test (#6188)
https://github.com/coder/coder/actions/runs/4165019548/jobs/7207442687
2023-02-14 00:46:05 +00:00
Marcin Tojek 271d68c862 feat: Disallow using legacy params with rich params (#5974)
* feat: Disallow using legacy params with rich params

* Fix

* nolint
2023-02-02 16:44:57 +01:00
Colin Adler d2ae16dd22 fix: routinely ping agent websocket to ensure liveness (#5824) 2023-01-23 20:05:29 +00:00