Commit Graph

381 Commits

Author SHA1 Message Date
Kyle Carberry 8ea956fc11 feat: add app status tracking to the backend (#17163)
This does ~95% of the backend work required to integrate the AI work.

Most left to integrate from the tasks branch is just frontend, which
will be a lot smaller I believe.

The real difference between this branch and that one is the abstraction
-- this now attaches statuses to apps, and returns the latest status
reported as part of a workspace.

This change enables us to have a similar UX to in the tasks branch, but
for agents other than Claude Code as well. Any app can report status
now.
2025-03-31 10:55:44 -04:00
Cian Johnston 06e5d9ef21 feat(coderd): add webpush package (#17091)
* Adds `codersdk.ExperimentWebPush` (`web-push`)
* Adds a `coderd/webpush` package that allows sending native push
notifications via `github.com/SherClockHolmes/webpush-go`
* Adds database tables to store push notification subscriptions.
* Adds an API endpoint that allows users to subscribe/unsubscribe, and
send a test notification (404 without experiment, excluded from API docs)
* Adds server CLI command to regenerate VAPID keys (note: regenerating
the VAPID keypair requires deleting all existing subscriptions)

---------

Co-authored-by: Kyle Carberry <kyle@carberry.com>
2025-03-27 10:03:53 +00:00
Vincent Vielle ddb06741c9 chore: improve dormant workspace notification wording (#17100)
Related to #17099
2025-03-26 15:54:03 +01:00
Jon Ayers 17ddee05e5 chore: update golang to 1.24.1 (#17035)
- Update go.mod to use Go 1.24.1
- Update GitHub Actions setup-go action to use Go 1.24.1
- Fix linting issues with golangci-lint by:
  - Updating to golangci-lint v1.57.1 (more compatible with Go 1.24.1)

🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <claude@anthropic.com>
2025-03-26 01:56:39 -05:00
brettkolodny cf10d98aab fix: improve error message when deleting organization with resources (#17049)
Closes
[coder/internal#477](https://github.com/coder/internal/issues/477)

![Screenshot 2025-03-21 at 11 25
57 AM](https://github.com/user-attachments/assets/50cc03e9-395d-4fc7-8882-18cb66b1fac9)

I'm solving this issue in two parts:

1. Updated the postgres function so that it doesn't omit 0 values in the
error
2. Created a new query to fetch the number of resources associated with
an organization and using that information to provider a cleaner error
message to the frontend

> **_NOTE:_** SQL is not my strong suit, and the code was created with
the help of AI. So I'd take extra time looking over what I wrote there
2025-03-25 15:31:24 -04:00
Mathias Fredriksson 5c8cac9fb7 feat: add name to workspace agent devcontainers (#17089)
In the presence of multiple devcontainers, it would be nice to
differentiate them by name. This change inherits the resource name from
terraform.

Refs #17076
2025-03-25 12:59:20 +00:00
Danny Kopping 4c33846f6d chore: add prebuilds system user (#16916)
Pre-requisite for https://github.com/coder/coder/pull/16891

Closes https://github.com/coder/internal/issues/515

This PR introduces a new concept of a "system" user.

Our data model requires that all workspaces have an owner (a `users`
relation), and prebuilds is a feature that will spin up workspaces to be
claimed later by actual users - and thus needs to own the workspaces in
the interim.

Naturally, introducing a change like this touches a few aspects around
the codebase and we've taken the approach _default hidden_ here; in
other words, queries for users will by default _exclude_ all system
users, but there is a flag to ensure they can be displayed. This keeps
the changeset relatively small.

This user has minimal permissions (it's equivalent to a `member` since
it has no roles). It will be associated with the default org in the
initial migration, and thereafter we'll need to somehow ensure its
membership aligns with templates (which are org-scoped) for which it'll
need to provision prebuilds; that's a solution we'll have in a
subsequent PR.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: Sas Swart <sas.swart.cdk@gmail.com>
2025-03-25 12:18:06 +00:00
Vincent Vielle 7b65422ef3 fix: change notifications actions url (#17083)
Related to #17082

Some notifications ( workspace created and workspace manually updated )
are using wrong variables to build the Action URL. Fixing it.
2025-03-25 11:29:02 +01:00
ケイラ 5b3eda6719 chore: persist template import terraform plan in postgres (#17012) 2025-03-24 10:01:50 -06:00
Vincent Vielle fe24a7a4a8 feat(coderd): remove greetings from notifications templates (#16991)
This PR aimes to [fix this
issue](https://github.com/coder/internal/issues/448) -

The main idea is to remove greetings from templates stored in the DB -
and instead push it into the template for require methods - for now
SMTP.
2025-03-21 16:05:08 +01:00
Hugo Dutka de6080c46d chore: update comment on the users.github_com_user_id field (#17037)
Follow up to https://github.com/coder/coder/pull/17029.
2025-03-21 13:31:17 +01:00
Mathias Fredriksson 69ba27e347 feat: allow specifying devcontainer on agent in terraform (#16997)
This change allows specifying devcontainers in terraform and plumbs it
through to the agent via agent manifest.

This will be used for autostarting devcontainers in a workspace.

Depends on coder/terraform-provider-coder#368
Updates #16423
2025-03-20 19:09:39 +02:00
Mathias Fredriksson 72d9876c76 fix(coderd/workspaceapps): prevent race in workspace app audit session updates (#17020)
Fixes coder/internal#520
2025-03-20 14:10:45 +00:00
Sas Swart a3f6308006 fix: rewrite login type migrations (#16978)
When trying to add [system
users](https://github.com/coder/coder/pull/16916), we discovered an
issue in two migrations that added values to the login_type enum.
After some
[consideration](https://github.com/coder/coder/pull/16916#discussion_r1998758887),
we decided to retroactively correct them.
2025-03-18 14:47:30 +02:00
Mathias Fredriksson de41bd6b95 feat: add support for workspace app audit (#16801)
This change adds support for workspace app auditing.

To avoid audit log spam, we introduce the concept of app audit sessions.
An audit session is unique per workspace app, user, ip, user agent and
http status code. The sessions are stored in a separate table from audit
logs to allow use-case specific optimizations. Sessions are ephemeral
and the table does not function as a log.

The logic for auditing is placed in the DBTokenProvider for workspace
apps so that wsproxies are included.

This is the final change affecting the API fo #15139.

Updates #15139
2025-03-18 13:50:52 +02:00
Danny Kopping b16275b7cd chore: fix regex bug in migration number fixer (#16822)
This fixes a slight regex bug on Bash 5, where `[:/]` would only match
`:` but not both `:/`.

```bash
$ git remote -v | grep "github.com[:/]coder/coder.*(fetch)" | cut -f1

$ git remote -v | grep "github.com[:/]*coder/coder.*(fetch)" | cut -f1
origin
```

The former will actually cause the whole script to bork because of
`pipefail`, since `grep` exits 1.

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-03-06 12:21:14 +02:00
Vincent Vielle 0c27f04bc7 fix(coderd): fix migration number overlapping (#16819)
Due to the [merge of this PR](https://github.com/coder/coder/pull/16764)
- two migration are overlapping in term of numbers - should increase
migration number of notifications.
2025-03-05 23:13:42 +01:00
Vincent Vielle 522181fead feat(coderd): add new dispatch logic for coder inbox (#16764)
This PR is [resolving the dispatch part of Coder
Inbocx](https://github.com/coder/internal/issues/403).

Since the DB layer has been merged - we now want to insert notifications
into Coder Inbox in parallel of the other delivery target.

To do so, we push two messages instead of one using the `Enqueue`
method.
2025-03-05 22:43:18 +01:00
ケイラ 9041646b81 chore: add "user_configs" db table (#16564) 2025-03-05 10:46:03 -07:00
Yevhenii Shcherbina b85ba586ee fix(coderd/database): consider tag sets when calculating queue position (#16685)
Relates to https://github.com/coder/coder/issues/15843

## PR Contents

- Reimplementation of the `GetProvisionerJobsByIDsWithQueuePosition` SQL
query to **take into account** provisioner job tags and provisioner
daemon tags.
- Unit tests covering different **tag sets**, **job statuses**, and
**job ordering** scenarios.

## Notes

- The original row order is preserved by introducing the `ordinality`
field.
- Unnecessary rows are filtered as early as possible to ensure that
expensive joins operate on a smaller dataset.
- A "fake" join with `provisioner_jobs` is added at the end to ensure
`sqlc.embed` compiles successfully.
- **Backward compatibility is preserved**—only the SQL query has been
updated, while the Go code remains unchanged.
2025-03-03 10:02:18 -05:00
Eng Zer Jun 04c33968cf refactor: replace golang.org/x/exp/slices with slices (#16772)
The experimental functions in `golang.org/x/exp/slices` are now
available in the standard library since Go 1.21.

Reference: https://go.dev/doc/go1.21#slices

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-03-04 00:46:49 +11:00
Vincent Vielle c074f77a4f feat: add notifications inbox db (#16599)
This PR is linked [to the following
issue](https://github.com/coder/internal/issues/334).

The objective is to create the DB layer and migration for the new `Coder
Inbox`.
2025-03-03 10:12:48 +01:00
Yevhenii Shcherbina 98dfc70f31 fix(coderd/database): remove linux build tags from db package (#16633)
Remove linux build tags from database package to make sure we can run
tests on Mac OS.
2025-02-25 11:39:37 -05:00
Jaayden Halko 546a549dcf feat: enable soft delete for organizations (#16584)
- Add deleted column to organizations table
- Add trigger to check for existing workspaces, templates, groups and
members in a org before allowing the soft delete

---------

Co-authored-by: Steven Masley <stevenmasley@gmail.com>
Co-authored-by: Steven Masley <Emyrk@users.noreply.github.com>
2025-02-24 12:59:41 -05:00
Danielle Maywood d2419c89ac feat: add tool to send a test notification (#16611)
Relates to https://github.com/coder/coder/issues/16463

Adds a CLI command, and API endpoint, to trigger a test notification for
administrators of a deployment.
2025-02-19 13:08:38 +00:00
Danielle Maywood d6b9806098 chore: implement oom/ood processing component (#16436)
Implements the processing logic as set out in the OOM/OOD RFC.
2025-02-17 16:56:52 +00:00
Mathias Fredriksson b5329ae1cd feat: add workspace agent connect and app open audit types (#16493)
This commit adds new audit resource types for workspace agents and
workspace apps, as well as connect/disconnect and open/close actions.

The idea is that we will log new audit events for connecting to the
agent via SSH/editor.

Likewise, we will log openings of `coder_app`s.

This change also introduces support for filtering by `request_id`.

Updates #15139
2025-02-17 13:02:30 +00:00
Sas Swart 71cbf735e5 feat(coderd): add support for presets to the coder API (#16526)
This pull request builds on the existing migrations and queries to add
support for presets to the coder API.
2025-02-12 14:41:14 +02:00
Sas Swart 34b46f9205 feat(coderd/database): add support for presets (#16509)
This pull requests adds the necessary migrations and queries to support
presets within the coderd database. Future PRs will build functionality
to the provisioners and the frontend.
2025-02-11 13:55:09 +02:00
Danielle Maywood dbad69dbd9 chore: add workspace oom/ood notification templates (#16250) 2025-02-04 19:25:18 +00:00
Vincent Vielle 7cbd77fd94 feat: improve resources_monitoring for OOM & OOD monitoring (#16241)
As requested for [this
issue](https://github.com/coder/internal/issues/245) we need to have a
new resource `resources_monitoring` in the agent.

It needs to be parsed from the provisioner and inserted into a new db
table.
2025-02-04 18:45:33 +01:00
Hugo Dutka 2ace044e0b chore: track the first time html is served in telemetry (#16334)
Addresses https://github.com/coder/nexus/issues/175.

## Changes

- Adds the `telemetry_items` database table. It's a key value store for
telemetry events that don't fit any other database tables.
- Adds a telemetry report when HTML is served for the first time in
`site.go`.
2025-01-31 13:55:46 +01:00
Mathias Fredriksson c069563af1 test: fix use of t.Logf where t.Log would suffice (#16328) 2025-01-29 14:35:04 +00:00
Steven Masley f34e6fd92c chore: implement 'use' verb to template object, read has less scope now (#16075)
Template `use` is now a verb.
- Template admins can `use` all templates (org template admins same in
org)
- Members get the `use` perm from the `everyone` group in the
`group_acl`.
2025-01-17 11:55:41 -06:00
Mathias Fredriksson 071bb26018 feat(coderd): add endpoint to list provisioner daemons (#16028)
Updates #15190
Updates #15084
Supersedes #15940
2025-01-14 16:40:26 +00:00
Danielle Maywood 009069cd47 feat: allow notification templates to be disabled by default (#16093)
Change as part of https://github.com/coder/coder/pull/16071

It has been decided that we want to be able to have some notification
templates be disabled _by default_
https://github.com/coder/coder/pull/16071#issuecomment-2580757061.

This adds a new column (`enabled_by_default`) to
`notification_templates` that defaults to `TRUE`. It also modifies the
`inhibit_enqueue_if_disabled` function to reject notifications for
templates that have `enabled_by_default = FALSE` with the user not
explicitly enabling it.
2025-01-13 15:01:47 +00:00
Sas Swart 4543b21b7c feat(coderd/database): track user status changes over time (#16019)
RE: https://github.com/coder/coder/issues/15740,
https://github.com/coder/coder/issues/15297

In order to add a graph to the coder frontend to show user status over
time as an indicator of license usage, this PR adds the following:

* a new `api.insightsUserStatusCountsOverTime` endpoint to the API
* which calls a new `GetUserStatusCountsOverTime` query from postgres
* which relies on two new tables `user_status_changes` and
`user_deleted`
* which are populated by a new trigger and function that tracks updates
to the users table

The chart itself will be added in a subsequent PR

---------

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2025-01-13 13:08:16 +02:00
Cian Johnston 7b88776403 chore(testutil): add testutil.GoleakOptions (#16070)
- Adds `testutil.GoleakOptions` and consolidates existing options to
this location
- Pre-emptively adds required ignore for this Dependabot PR to pass CI
https://github.com/coder/coder/pull/16066
2025-01-08 15:38:37 +00:00
Vincent Vielle 08463c27d8 feat: add OpenIn option to coder_app (#15743)
This PR is the coder/coder part of [the open_in parameter
issue](https://github.com/coder/terraform-provider-coder/issues/297)
aiming to add a new optional parameter to choose how to open modules.

This PR is heavily linked [to this
PR](https://github.com/coder/terraform-provider-coder/pull/321).

ℹ️ For now, some integrations tests can not be pushed as it requires a
release on the terraform-provider repo.
2025-01-03 11:27:02 +01:00
Steven Masley 73ec6b2635 chore: audit log entries for all idp sync changes (#15919) 2025-01-02 15:02:04 -06:00
Danielle Maywood f3fe3bc785 feat: notify on workspace update (#15979)
Relates to https://github.com/coder/coder/issues/15845

When the `/workspace/<name>/builds` endpoint is hit, we check if the
requested template version is different to the previously used template
version. If these values differ, we can assume that the workspace has
been manually updated and send the appropriate notification. Automatic
updates happen in the lifecycle executor and bypasses this endpoint
entirely.
2025-01-02 12:19:34 +00:00
Danielle Maywood f0e81ab455 feat: notify on workspace creation (#15934) 2024-12-20 13:53:10 +00:00
Danielle Maywood e21a301682 fix: make GetWorkspacesEligibleForTransition return even less false positives (#15594)
Relates to https://github.com/coder/coder/issues/15082

Further to https://github.com/coder/coder/pull/15429, this reduces the
amount of false-positives returned by the 'is eligible for autostart'
part of the query. We achieve this by calculating the 'next start at'
time of the workspace, storing it in the database, and using it in our
`GetWorkspacesEligibleForTransition` query.

The prior implementation of the 'is eligible for autostart' query would
return _all_ workspaces that at some point in the future _might_ be
eligible for autostart. This now ensures we only return workspaces that
_should_ be eligible for autostart.

We also now pass `currentTick` instead of `t` to the
`GetWorkspacesEligibleForTransition` query as otherwise we'll have one
round of workspaces that are skipped by `isEligibleForTransition` due to
`currentTick` being a truncated version of `t`.
2024-12-02 21:02:36 +00:00
Hugo Dutka b830c05e3e chore: track usage of built-in example templates (#15671)
Addresses https://github.com/coder/nexus/issues/99.

Changes:
- Save the id of the built-in example template used to create a template
version in the database
- Include the example id in telemetry
2024-11-27 20:01:08 +01:00
Hugo Dutka aa0dc2daa1 chore: track terraform modules in telemetry (#15450)
Addresses https://github.com/coder/nexus/issues/35.

This PR:

- Adds a `workspace_modules` table to track modules used by the
Terraform provisioner in provisioner jobs.
- Adds a `module_path` column to the `workspace_resources` table,
allowing to identify which module a resource originates from.
- Starts pushing this new information into telemetry.

For the person reviewing this PR, do not fret about the 1,500 new lines
- ~1,000 of them are auto-generated.
2024-11-16 21:56:19 +01:00
Sas Swart 450c72f95c chore(coderd/database): fix duplicate migration numbers (#15533) 2024-11-15 11:39:05 +00:00
Sas Swart dbf41a1160 chore(coderd/database): fix duplicate migration numbers (#15530)
Renaming migrations to avoid duplicate numbering
2024-11-15 10:55:47 +00:00
Sas Swart 814dd6f854 feat(coderd): update API to allow filtering provisioner daemons by tags (#15448)
This PR provides new parameters to an endpoint that will be necessary
for #15048
2024-11-15 11:33:22 +02:00
Steven Masley b6d0b7713a chore: implement user link claims as a typed golang object (#15502)
Move claims from a `debug` column to an actual typed column to be used.
This does not functionally change anything, it just adds some Go typing to build
on.
2024-11-14 10:05:44 -06:00
Hugo Dutka 1bfa7d42e8 chore: add postgres template caching for tests (#15336)
This PR is the first in a series aimed at closing
[#15109](https://github.com/coder/coder/issues/15109).

### Changes

- **Template Database Creation:**  
`dbtestutil.Open` now has the ability to create a template database if
none is provided via `DB_FROM`. The template database’s name is derived
from a hash of the migration files, ensuring that it can be reused
across tests and is automatically updated whenever migrations change.

- **Optimized Database Handling:**  
Previously, `dbtestutil.Open` would spin up a new container for each
test when `DB_FROM` was unset. Now, it first checks for an active
PostgreSQL instance on `localhost:5432`. If none is found, it creates a
single container that remains available for subsequent tests,
eliminating repeated container startups.

These changes address the long individual test times (10+ seconds)
reported by some users, likely due to the time Docker took to start and
complete migrations.
2024-11-04 17:23:31 +01:00