Commit Graph
60 Commits
Author SHA1 Message Date
ケイラ 9fc3329575 feat: persist app groups in the database (#17977) 2025-05-27 13:13:08 -06:00
Susana Ferreira 6f6e73af03 feat: implement expiration policy logic for prebuilds (#17996)
## Summary 

This PR introduces support for expiration policies in prebuilds. The TTL
(time-to-live) is retrieved from the Terraform configuration
([terraform-provider-coder
PR](https://github.com/coder/terraform-provider-coder/pull/404)):
```
prebuilds = {
	  instances = 2
	  expiration_policy {
		  ttl = 86400
	  }
  }
```
**Note**: Since there is no need for precise TTL enforcement down to the
second, in this implementation expired prebuilds are handled in a single
reconciliation cycle: they are deleted, and new instances are created
only if needed to match the desired count.

## Changes

* The outcome of a reconciliation cycle is now expressed as a slice of
reconciliation actions, instead of a single aggregated action.
* Adjusted reconciliation logic to delete expired prebuilds and
guarantee that the number of desired instances is correct.
* Updated relevant data structures and methods to support expiration
policies parameters.
* Added documentation to `Prebuilt workspaces` page
* Update `terraform-provider-coder` to version 2.5.0:
https://github.com/coder/terraform-provider-coder/releases/tag/v2.5.0

Depends on: https://github.com/coder/terraform-provider-coder/pull/404
Fixes: https://github.com/coder/coder/issues/17916
2025-05-26 20:31:24 +01:00
Danny Kopping cf98268031 chore: push proto changes to v1.6 (#17874)
`v1.5` is going out with release `v2.22`

I had to reorder `module_files` and `resource_replacements` because of
this.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-05-16 11:27:41 +02:00
Thomas Kosiewski 1bacd82e80 feat: add API key scope to restrict access to user data (#17692) 2025-05-15 15:32:52 +01:00
Danny Kopping 6e967780c9 feat: track resource replacements when claiming a prebuilt workspace (#17571)
Closes https://github.com/coder/internal/issues/369

We can't know whether a replacement (i.e. drift of terraform state
leading to a resource needing to be deleted/recreated) will take place
apriori; we can only detect it at `plan` time, because the provider
decides whether a resource must be replaced and it cannot be inferred
through static analysis of the template.

**This is likely to be the most common gotcha with using prebuilds,
since it requires a slight template modification to use prebuilds
effectively**, so let's head this off before it's an issue for
customers.

Drift details will now be logged in the workspace build logs:


![image](https://github.com/user-attachments/assets/da1988b6-2cbe-4a79-a3c5-ea29891f3d6f)

Plus a notification will be sent to template admins when this situation
arises:


![image](https://github.com/user-attachments/assets/39d555b1-a262-4a3e-b529-03b9f23bf66a)

A new metric - `coderd_prebuilt_workspaces_resource_replacements_total`
- will also increment each time a workspace encounters replacements.

We only track _that_ a resource replacement occurred, not how many. Just
one is enough to ruin a prebuild, but we can't know apriori which
replacement would cause this.
For example, say we have 2 replacements: a `docker_container` and a
`null_resource`; we don't know which one might
cause an issue (or indeed if either would), so we just track the
replacement.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-05-14 14:52:22 +02:00
Sas SwartandDanny Kopping 425ee6fa55 feat: reinitialize agents when a prebuilt workspace is claimed (#17475)
This pull request allows coder workspace agents to be reinitialized when
a prebuilt workspace is claimed by a user. This facilitates the transfer
of ownership between the anonymous prebuilds system user and the new
owner of the workspace.

Only a single agent per prebuilt workspace is supported for now, but
plumbing has already been done to facilitate the seamless transition to
multi-agent support.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
Co-authored-by: Danny Kopping <dannykopping@gmail.com>
2025-05-14 14:15:36 +02:00
Steven Masley 398b999d8f chore: pass previous values into terraform apply (#17696)
Pass previous workspace build parameter values into the terraform
`plan/apply`. Enforces monotonicity in terraform as well as `coderd`.
2025-05-12 15:32:00 -05:00
ケイラ d0ab91c16f fix: reduce size of terraform modules archive (#17749) 2025-05-12 13:50:07 -06:00
Danny Kopping af2941bb92 feat: add is_prebuild_claim to distinguish post-claim provisioning (#17757)
Used in combination with
https://github.com/coder/terraform-provider-coder/pull/396

This is required by both https://github.com/coder/coder/pull/17475 and
https://github.com/coder/coder/pull/17571

Operators may need to conditionalize their templates to perform certain
operations once a prebuilt workspace has been claimed. This value will
**only** be set once a claim takes place and a subsequent `terraform
apply` occurs. Any `terraform apply` runs thereafter will be
indistinguishable from a normal run on a workspace.

---------

Signed-off-by: Danny Kopping <dannykopping@gmail.com>
2025-05-12 14:19:03 +00:00
Jon Ayers a9f1a6b2a2 fix: revert fix: persist terraform modules during template import (#17665) (#17734)
This reverts commit ae3d90b057.
2025-05-08 22:03:08 -04:00
ケイラ ae3d90b057 fix: persist terraform modules during template import (#17665) 2025-05-08 16:13:46 -06:00
ケイラ 5b3eda6719 chore: persist template import terraform plan in postgres (#17012) 2025-03-24 10:01:50 -06:00
Mathias Fredriksson 69ba27e347 feat: allow specifying devcontainer on agent in terraform (#16997)
This change allows specifying devcontainers in terraform and plumbs it
through to the agent via agent manifest.

This will be used for autostarting devcontainers in a workspace.

Depends on coder/terraform-provider-coder#368
Updates #16423
2025-03-20 19:09:39 +02:00
Sas Swart 46e04c68e3 feat(provisioner): add support for presets to coder provisioners (#16574)
This pull request adds support for presets to coder provisioners.
If a template defines presets using a compatible version of the
provider, then this PR will allow those presets to be persisted to the
control plane database for use in workspace creation.
2025-02-17 13:00:44 +02:00
Vincent Vielle 7cbd77fd94 feat: improve resources_monitoring for OOM & OOD monitoring (#16241)
As requested for [this
issue](https://github.com/coder/internal/issues/245) we need to have a
new resource `resources_monitoring` in the agent.

It needs to be parsed from the provisioner and inserted into a new db
table.
2025-02-04 18:45:33 +01:00
Thomas Kosiewski 1336925c9f feat(flake.nix): switch dogfood dev image to buildNixShellImage from dockerTools (#16223)
Replace Depot build action with Nix for Nix dogfood image builds

The dogfood Nix image is now built using Nix's native container tooling instead of Depot. This change:

- Adds Nix setup steps to the GitHub Actions workflow
- Removes the Dockerfile.nix in favor of a Nix-native container build
- Updates the flake.nix to support building Docker images
- Introduces a hash file to track Nix-related changes
- Updates the vendorHash for Go dependencies

Change-Id: I4e011fe3a19d9a1375fbfd5223c910e59d66a5d9
Signed-off-by: Thomas Kosiewski <tk@coder.com>
2025-01-28 16:38:37 +01:00
Vincent Vielle 08463c27d8 feat: add OpenIn option to coder_app (#15743)
This PR is the coder/coder part of [the open_in parameter
issue](https://github.com/coder/terraform-provider-coder/issues/297)
aiming to add a new optional parameter to choose how to open modules.

This PR is heavily linked [to this
PR](https://github.com/coder/terraform-provider-coder/pull/321).

ℹ️ For now, some integrations tests can not be pushed as it requires a
release on the terraform-provider repo.
2025-01-03 11:27:02 +01:00
Hugo Dutka aa0dc2daa1 chore: track terraform modules in telemetry (#15450)
Addresses https://github.com/coder/nexus/issues/35.

This PR:

- Adds a `workspace_modules` table to track modules used by the
Terraform provisioner in provisioner jobs.
- Adds a `module_path` column to the `workspace_resources` table,
allowing to identify which module a resource originates from.
- Starts pushing this new information into telemetry.

For the person reviewing this PR, do not fret about the 1,500 new lines
- ~1,000 of them are auto-generated.
2024-11-16 21:56:19 +01:00
Danny Kopping 6960d194ae feat: add provisioning timings to understand slow build times (#14274) 2024-08-21 14:18:58 +02:00
Marcin Tojek b8b80fe6d2 feat: store coder_workspace_tags in the database (#13294) 2024-05-20 13:30:19 +00:00
Kayla Washburn-Love 475c3650ca feat: add support for optional external auth providers (#12021) 2024-02-21 11:18:38 -07:00
Cian Johnston 643c3ee54b refactor(provisionerd): move provisionersdk.VersionCurrent -> provisionerdproto.VersionCurrent (#12225) 2024-02-20 12:44:19 +00:00
Kyle Carberry 8abca9bea7 chore: rename git_auth to external_auth in our schema (#9935)
* chore: rename `git_auth` to `external_auth` in our schema

We're changing Git auth to be external auth. It will support
any OAuth2 or OIDC provider.

To split up the larger change I want to contribute the schema
changes first, and I'll add the feature itself in another PR.

* Fix names

* Fix outdated view

* Rename some additional places

* Fix sort order

* Fix template versions auth route

* Fix types

* Fix dbauthz
2023-09-29 19:13:20 +00:00
Spike Curtis 375c70d141 feat: integrate Acquirer for provisioner jobs (#9717)
* chore: add Acquirer to provisionerdserver pkg

Signed-off-by: Spike Curtis <spike@coder.com>

* code review improvements & fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* feat: integrate Acquirer for provisioner jobs

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix imports, whitespace

Signed-off-by: Spike Curtis <spike@coder.com>

* provisionerdserver always closes; remove poll interval from playwright

Signed-off-by: Spike Curtis <spike@coder.com>

* post jobs outside transactions

Signed-off-by: Spike Curtis <spike@coder.com>

* graceful shutdown in test

Signed-off-by: Spike Curtis <spike@coder.com>

* Mark AcquireJob deprecated

Signed-off-by: Spike Curtis <spike@coder.com>

* Graceful shutdown on all provisionerd tests

Signed-off-by: Spike Curtis <spike@coder.com>

* Deprecate, not remove CLI flags

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-09-19 10:25:57 +04:00
Spike Curtis 60d5002eb6 refactor: change template archive extraction to be on provisioner (#9264)
* refactor provisionersdk protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisioners to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisionerd to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor tests & proto renames

* Fixes from self-review

Signed-off-by: Spike Curtis <spike@coder.com>

* appease fmt & link

Signed-off-by: Spike Curtis <spike@coder.com>

* code review fixes & e2e fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* More fmt

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* new gen; use uuid for session workdir

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert nix-based gen CI task until dogfood is on nix

Signed-off-by: Spike Curtis <spike@coder.com>

* revert deleting dogfood Docker stuff

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert "revert deleting dogfood Docker stuff"

This reverts commit 9762158167.

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-25 06:10:15 +00:00
Jon Ayers 7ddb216d87 chore: revert nix-related CI changes (#9321)
* chore: revert nix-related CI changes

- Reverts using nix to run CI-dependencies.
- Running 'make gen' in a dogfood workspace resulted in inconsistent
  results for protobuf-related files making it difficult to pass CI.
  This PR imports the minimum changes necessary to make CI compatible
  with dogfood.
2023-08-24 21:26:30 -05:00
Kyle Carberry 22e781eced chore: add /v2 to import module path (#9072)
* chore: add /v2 to import module path

go mod requires semantic versioning with versions greater than 1.x

This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```

Migrate generated files to import /v2

* Fix gen
2023-08-18 18:55:43 +00:00
Kyle Carberry c0a78533bf chore: reduce build time by converting docker image to build with nix (#9099)
* Add Docker image

* Try building the container in CI

* Uncomment testing lines

* Trim image step

* Hit 'em with a Docker load

* Oopsie!

* Add a tag and push!

* Fix image name 🤦

* Fix sudo

* Fix target name

* Build and push

* Fix login to docker hub

* Revert to Docker push

* Fix PATH

* Fix SHA

* Fix ca certs

* Fix ca certs

* Fix coping files in

* Fix docker init

* Fix Docker group and init

* Add comments to our Nix

* Fix build stage

* Add some more comments

* Remove old dogfood image files

* Tag and push with branch name

* Fix tag passing

* Fix tag passing

* Remove old pull triggers

* Convert gen to use Nix

* Add protobuf to the flake

* Add prettier to the dev shell

* Swap to the faster Nix cache action

* Add the correct yq

* Fix gen

* Add make to the flake

* Update extensions
2023-08-17 21:19:36 -05:00
Colin Adler 7d43f4ceee chore: update dogfood image (#8114) 2023-06-20 15:08:15 -05:00
Marcin Tojek a7366a8b76 feat!: drop support for legacy parameters (#7663) 2023-06-02 11:16:46 +02:00
Marcin Tojek 4eb0baa849 chore: update protoc-gen-go (#7708) 2023-05-30 10:24:34 +02:00
Colin Adler 8bd9f9c351 feat: unified tracing between coderd<->provisionerd (#7370) 2023-05-03 23:02:35 +00:00
Marcin Tojek 0ba200c2a1 feat: Enable workspace debug logging (#6838)
* feat: Enable workspace debug logging

* Fix

* Fix

* Fix

* fix

* fix

* Enable RBAC

* unit tests

* Fix

* fix

* fix

* fix

* more tests

* fix: workspacebuild_test use roles

* fix: swagger comment

* fix: ctx.Done

* fix: address PR comments

* break loop
2023-03-30 16:00:33 +02:00
Kyle Carberry df31636e72 feat: pass access_token to coder_git_auth resource (#6713)
This allows template authors to leverage git auth to perform
custom actions, like clone repositories.
2023-03-22 19:37:08 +00:00
Marcin Tojek 3b87316ad7 feat: propagate job error codes (#6507)
* feat: propagate job error_code

* fix

* Fix

* Fix

* Fix

* add errors to typesGenerated

* Address PR comments

* Fix
2023-03-08 16:32:00 +01:00
Kyle Carberry 7f226d4f90 feat: add support for coder_git_auth data source (#6334)
* Add git auth providers schema

* Pipe git auth providers to the schema

* Add git auth providers to the API

* Add gitauth endpoint to query authenticated state

* Add endpoint to query git state

* Use BroadcastChannel to automatically authenticate with Git

* Add error validation for submitting the create workspace form

* Fix panic on template dry-run

* Add tests for the template version Git auth endpoint

* Show error if no gitauth is configured

* Add gitauth to cliui

* Fix unused method receiver

* Fix linting errors

* Fix dbauthz querier test

* Fix make gen

* Add JavaScript test for git auth

* Fix bad error message

* Fix provisionerd test race

See https://github.com/coder/coder/actions/runs/4277960646/jobs/7447232814

* Fix requested changes

* Add comment to CreateWorkspacePageView
2023-02-27 10:18:19 -06:00
Marcin Tojek 3b7b96ac28 feat: Expose managed variables via API (#6134)
* WIP

* hcl

* useManagedVariables

* fix

* Fix

* Fix

* fix

* go:build

* Fix

* fix: bool flag

* Insert template variables

* API

* fix

* Expose via API

* More wiring

* CLI for testing purposes

* WIP

* Delete FIXME

* planVars

* WIP

* WIP

* UserVariableValues

* no dry run

* Dry run

* Done FIXME

* Fix

* Fix: CLI

* Fix: migration

* API tests

* Test info

* Tests

* More tests

* fix: lint

* Fix: authz

* Address PR comments

* Fix

* fix

* fix
2023-02-15 18:24:15 +01:00
Marcin Tojek bbb208e29c feat: Add CLI support for workspace build parameters (#5768)
* WIP

* WIP

* CLI: handle workspace build parameters

* fix: golintci

* Fix: dry run

* fix

* CLI: is mutable

* coderd: mutable

* fix: golanci

* fix: richParameterFile

* CLI: create unit tests

* CLI: update test

* Fix

* fix: order

* fix
2023-01-23 15:01:22 +01:00
Marcin Tojek 6ebadabe4e feat: Add basic support for rich parameters to coderd and provisionerd (#5710) 2023-01-17 11:22:11 +01:00
Ammar Bandukwala 97dbd4dc5d Implement Quotas v3 (#5012)
* provisioner/terraform: add cost to resource_metadata

* provisionerd/runner: use Options struct

* Complete provisionerd implementation

* Add quota_allowance to groups

* Combine Quota and RBAC licenses

* Add Opts to InTx
2022-11-14 17:57:33 +00:00
Ammar Bandukwala 30f8fd9b95 Daily Active User Metrics (#3735)
* agent: add StatsReporter

* Stabilize protoc
2022-09-01 14:58:23 -05:00
Kyle Carberry b0fe9bcdd1 chore: Upgrade to Go 1.19 (#3617)
This is required as part of #3505.
2022-08-21 22:32:53 +00:00
Dean Sheather 6be8a373e0 feat: run a terraform plan before creating workspaces with the given template parameters (#1732) 2022-06-02 00:44:53 +10:00
David Wahler 0c4a65b113 fix: manually fix coderd/database/dump.sql and make style/gen check run more reliably (#1607) 2022-05-19 22:37:22 +00:00
Colin Adler 98ccd0eb89 feat: add README parsing to template versions (#1500) 2022-05-17 15:00:48 -05:00
Steven Masley da3681246e chore: Bump protoc to 3.20.0 (#1104)
* chore: Bump protoc to 3.20.0
* Make gen with 3.20.0 protoc
2022-04-23 01:53:22 +00:00
Kyle Carberry 02ad3f14f5 chore: Rename Projects to Templates (#880)
Customer feedback indicated projects was a confusing name.
After querying the team internally, it seemed unanimous
that it is indeed a confusing name.

Here's for a lil less confusion @ashmeer7 🥂
2022-04-06 12:42:40 -05:00
Kyle Carberry b33dec9d38 feat: Add stage to build logs (#577)
* feat: Add stage to build logs

This adds a stage property to logs, and refactors the job logs
cliui.

It also adds tests to the cliui for build logs!

* Fix comments
2022-03-28 18:43:22 +00:00
Kyle Carberry c451f4e685 feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00
Colin Adler 706e60bb3b chore: pluralize table names (#463) 2022-03-17 18:10:46 -05:00