Commit Graph
73 Commits
Author SHA1 Message Date
Kyle Carberry b0fe9bcdd1 chore: Upgrade to Go 1.19 (#3617)
This is required as part of #3505.
2022-08-21 22:32:53 +00:00
Mathias Fredriksson 3fdae47b87 fix: Shadow err in TestProvision_Cancel to fix test race (#3579)
Fixes #3574
2022-08-19 11:56:28 +03:00
Mathias Fredriksson f1423450bd fix: Allow terraform provisions to be gracefully cancelled (#3526)
* fix: Allow terraform provisions to be gracefully cancelled

This change allows terraform commands to be gracefully cancelled on
Unix-like platforms by signaling interrupt on provision cancellation.

One implementation detail to note is that we do not necessarily kill a
running terraform command immediately even if the stream is closed. The
reason for this is to allow for graceful cancellation even in such an
event. Currently the timeout is set to 5 minutes by default.

Related: #2683

The above issue may be partially or fully fixed by this change.

* fix: Remove incorrect minimumTerraformVersion variable

* Allow init to return provision complete response
2022-08-18 17:03:55 +03:00
Mathias Fredriksson fb9fca8bc9 fix: Ensure terraform tests have a cache path and logger (#3161)
* fix: Ensure terraform tests have a cache path and logger

* fix: Protect against concurrent `terraform init`
2022-08-04 20:37:07 +03:00
Kyle Carberry 303b280e0e fix: Associate spot instances with their instance IDs for auth (#3383)
Fixes #2162.
2022-08-04 10:20:56 -05:00
Kyle Carberry 69664ed168 fix: Use "virtual_machine_id" for instance identity with Azure (#3355)
This was using the wrong property, causing automatic auth to break.
2022-08-03 12:19:13 -05:00
David Wahler 8a2811210a feat: Add backend API support for resource metadata (#3242)
* Initial support for metadata in provisioner API and Terraform provisioner

* add support for nullable metadata fields

* handle metadata fields in provisionerd and API
2022-08-01 16:53:05 -05:00
Cian Johnston 6377f17fda chore: update terraform to 1.2.1 (#3243)
* chore: update terraform to 1.2.1

* allow terraform version equal to max
2022-07-27 17:11:38 +01:00
David Wahler fbd1a272fe fix: Fix dangling references in provisioner/terraform/testdata (#3193) 2022-07-26 12:04:21 -05:00
Ammar Bandukwala f28d14197a Rename default agent to "main" instead of "dev" (#3150)
Resolves #3143
2022-07-23 20:26:56 +00:00
David Wahler f7ea016494 Pass git configuration variables via terraform (#3034)
* Pass workspace owner email address to provisioner

* Remove owner_email and owner_username fields from agent metadata

* Add Git environment variables to example templates

* Remove "owner_name" field from provisioner metadata, use username instead

* Remove Git configuration from most templates, add documentation

* Proofreading/typo fixes from @mafredri

* Update example templates to latest version of terraform-provider-coder
2022-07-19 13:24:06 -05:00
David Wahler b5f5e909bd Return template parameters in consistent order (#2975)
* return parameters from Terraform provisioner in sorted order

* persist parameter indices in database and return them in correct order from API

* don't re-sort parameters by name when creating templates
2022-07-13 15:29:34 -05:00
Jon Ayers 2353687610 feat: unexpose coderdtest.NewWithAPI (#2613)
* feat: unexpose coderdtest.NewWithAPI
2022-06-27 13:50:52 -05:00
Kyle Carberry 33a04f661f fix: Check if start is nil before consuming in echo provisioner (#2686)
This caused a race seen here:
https://github.com/coder/coder/runs/7074123929?check_suite_focus=true#step:10:217
2022-06-27 09:57:37 -05:00
Abhineet Jain c6b1daabc5 feat: Download default terraform version when minor version mismatches (#1775) 2022-06-22 23:11:52 +00:00
Dean Sheather a9c166491d fix: synchronize terraform log output, fix init coloring (#2482) 2022-06-19 05:26:43 +10:00
Spike Curtis 54a585dbf6 Log provisioner outputs from TestProvision_ExtraEnv (#2427)
Signed-off-by: Spike Curtis <spike@coder.com>
2022-06-17 14:47:05 -07:00
Spike Curtis 552dad6919 Remove tfexec, allow TF_ environment vars and log them (#2264)
* Remove tfexec, allow TF_ environment vars and log them

Signed-off-by: Spike Curtis <spike@coder.com>

* fixup: commented code, long lines

Signed-off-by: Spike Curtis <spike@coder.com>

* rename executor methods to remove get

Signed-off-by: Spike Curtis <spike@coder.com>

* don't log terraform environment variables we don't know are safe

Signed-off-by: Spike Curtis <spike@coder.com>

* Disable linting of fake secret

Signed-off-by: Spike Curtis <spike@coder.com>

* drop parse support and move logger into terraform package

Signed-off-by: Spike Curtis <spike@coder.com>

* disable testpackage linter on internal package test

Signed-off-by: Spike Curtis <spike@coder.com>
2022-06-16 17:50:39 +00:00
Kyle Carberry 024ab6df57 fix: Use in-memory filesystem for echo provisioner tests (#2408)
* fix: Use in-memory filesystem for echo provisioner tests

This should reduce IO in CI to shave some time off tests!

* test: Increase timeouts to reduce flakes

It's difficult to understand what's timing out due to a lock
vs. taking a long time. This should help resolve! 🕵️
2022-06-16 15:09:22 +00:00
Dean Sheather 45eb1b4980 feat: improve terraform template parsing errors (#2331) 2022-06-16 04:12:17 +10:00
Colin Adler e3a1cd34b7 fix: ensure agentResource is non-nil (#2261) 2022-06-11 00:02:49 +00:00
Kyle Carberry f9290b016e fix: Use explicit resource order when assocating agents (#2219)
This cleans up agent association code to explicitly map a single
agent to a single resource. This will fix #1884, and unblock
a prospect from beginning a POC.
2022-06-10 15:47:36 +00:00
Kyle Carberry fff59ef6ad fix: Add tests for instance and app association (#2198)
This was regressed in #2187. There was bad testing around this
before, and this should prevent a similiar situation from happening
again!
2022-06-09 13:34:24 +00:00
Kyle Carberry 14701498c9 fix: Improve Terraform agent<->resource association testing (#2187) 2022-06-08 17:40:34 -05:00
Kyle Carberry 013f028e55 feat: Add workspace application support (#1773)
* feat: Add app support

This adds apps as a property to a workspace agent.

The resource is added to the Terraform provider here:
https://github.com/coder/terraform-provider-coder/pull/17

Apps will be opened in the dashboard or via the CLI
with `coder open <name>`. If `command` is specified, a
terminal will appear locally and in the web. If `target`
is specified, the browser will open to an exposed instance
of that target.

* Compare fields in apps test

* Update Terraform provider to use relative path

* Add some basic structure for routing

* chore: Remove interface from coderd and lift API surface

Abstracting coderd into an interface added misdirection because
the interface was never intended to be fulfilled outside of a single
implementation.

This lifts the abstraction, and attaches all handlers to a root struct
named `*coderd.API`.

* Add basic proxy logic

* Add proxying based on path

* Add app proxying for wildcards

* Add wsconncache

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* Add workspace route proxying endpoint

- Makes the workspace conn cache concurrency-safe
- Reduces unnecessary open checks in `peer.Channel`
- Fixes the use of a temporary context when dialing a workspace agent

* Add embed errors

* chore: Refactor site to improve testing

It was difficult to develop this package due to the
embed build tag being mandatory on the tests. The logic
to test doesn't require any embedded files.

* Add test for error handler

* Remove unused access url

* Add RBAC tests

* Fix dial agent syntax

* Fix linting errors

* Fix gen

* Fix icon required

* Adjust migration number

* Fix proxy error status code

* Fix empty db lookup
2022-06-04 15:13:37 -05:00
Dean Sheather 6be8a373e0 feat: run a terraform plan before creating workspaces with the given template parameters (#1732) 2022-06-02 00:44:53 +10:00
Abhineet JainandMathias Fredriksson c78f947e09 feat: Upgrade terraform version to 1.1.9 (#1745)
* upgrade terraform version to 1.1.9

* Fix docs typo

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2022-05-25 19:35:41 -04:00
Kyle Carberry 74329f479f fix: Use Terraform address to index resource + agent association (#1727)
Closes #1705.

There was an issue in the implementation brought by #1577 by not trimming
the array value when resources use counts. This should fix it, and adds
a test to be sure!
2022-05-24 20:00:34 -05:00
Cian Johnston c2f74f3cc2 chore: avoid concurrent usage of t.FailNow (#1683)
* chore: golangci: add linter rule to report usage of t.FailNow inside goroutines
* chore: avoid t.FailNow in goroutines to appease the race detector
2022-05-24 08:58:39 +01:00
Dean Sheather adb7d20c16 feat: skip terraform destroy if there is no state when deleting (#1594) 2022-05-20 14:07:23 +10:00
Cian Johnston 8814cb0722 Revert "fix: Use Terraform address to index resource + agent association (#1577)" (#1585)
This reverts commit f3fe2a08ce.
2022-05-19 12:18:40 +01:00
Kyle Carberry f3fe2a08ce fix: Use Terraform address to index resource + agent association (#1577)
This fixes resources created from Terraform modules not
properly being associated with an agent.

By not using the address, and resource identifiers prefixed
with `module.<name>` would be missed!
2022-05-18 16:26:08 -05:00
Kyle Carberry ba818b3a10 fix: Append Terraform module resources to list (#1539)
This was causing module resources to be skipped!
2022-05-17 19:07:20 -05:00
Colin Adler 98ccd0eb89 feat: add README parsing to template versions (#1500) 2022-05-17 15:00:48 -05:00
Kyle Carberry a2ba69dd28 fix: Parse resources from Terraform Modules (#1501)
Fixes when Terraform modules are used to primariy provision
infrastructure!
2022-05-16 20:56:50 -05:00
Kyle Carberry 43c6bff5ae fix: Use "terraform state pull" instead of "terraform show" (#1262)
Although the terraform-exec docs don't indicate this, the result of
"terraform show" isn't actually the state... it's a trimmed version
of the state that excludes resource identifiers, essentially removing
all state that did exist.

Tests will be written to ensure Terraform state reconciliation can occur.
This will happen in another PR, as dogfood is currently broken because of this.
2022-05-02 20:02:38 +00:00
Kyle Carberry d139a16446 fix: Allow remote state to be used with Terraform (#1242)
The Terraform Provisioner depended on the statefile content
being at a specific path, which disallowed the use of external
state providers. This fixes it!
2022-05-02 15:41:27 +00:00
Kyle Carberry 8701e0084c feat: Update Terraform provider to support "dir" in "coder_agent" (#1219)
This allows users to specify a starting directory for shell sessions.
2022-05-02 10:27:34 -05:00
Kyle Carberry b948f2dab5 fix: Use environment variables for agent authentication (#1238)
* fix: Update GIT_COMMITTER_NAME to use username

This was a mistake when adding the committer fields 🤦.

* fix: Use environment variables for agent authentication

Using files led to situations where running "coder server --dev" would
break `gitssh`. This is applicable in a production environment too. Users
should be able to log into another Coder deployment from their workspace.

Users can still set "CODER_URL" if they'd like with agent env vars!
2022-04-30 16:40:30 +00:00
Kyle Carberry 65d77383d0 fix: Allow nested Terraform resources (#1093)
This fixes the dependency tree by adding recursion. It
now finds indirect connections and associates it with
an agent.

An example is attached which surfaced this issue.
2022-04-20 12:28:48 -05:00
Kyle Carberry 19b4323512 feat: Allow workspace resources to attach multiple agents (#942)
This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.
2022-04-11 16:06:15 -05:00
Kyle Carberry 35a0acc9c6 fix: Disable Terraform plugin cache on Darwin (#927)
It was occasionally failing without any clear indication of
what to fix on our side. The plugins weren't being found
by Terraform.

We already disable this on Windows, so figured it's fine on
Darwin too considering most production deployments will be Linux.
2022-04-08 12:30:31 -05:00
Colin Adler dc46ff407b fix: ensure websocket close messages are truncated to 123 bytes (#779)
It's possible for websocket close messages to be too long, which cause
them to silently fail without a proper close message. See error below:

```
2022-03-31 17:08:34.862 [INFO]	(stdlib)	<close_notjs.go:72>	"2022/03/31 17:08:34 websocket: failed to marshal close frame: reason string max is 123 but got \"insert provisioner daemon:Cannot encode []database.ProvisionerType into oid 19098 - []database.ProvisionerType must implement Encoder or be converted to a string\" with length 161"
```
2022-04-01 18:17:45 +00:00
Kyle Carberry 6612e3c9c7 feat: Add config-ssh command (#735)
* feat: Add config-ssh command

Closes #254 and #499.

* Fix Windows support
2022-03-30 17:59:54 -05:00
Kyle Carberry 82dfd6c72f feat: Add UI for awaiting agent connections (#578)
* feat: Add stage to build logs

This adds a stage property to logs, and refactors the job logs
cliui.

It also adds tests to the cliui for build logs!

* feat: Add stage to build logs

This adds a stage property to logs, and refactors the job logs
cliui.

It also adds tests to the cliui for build logs!

* feat: Add config-ssh and tests for resiliency

* Rename "Echo" test to "ImmediateExit"

* Fix Terraform resource agent association

* Fix logs post-cancel

* Fix select on Windows

* Remove terraform init logs

* Move timer into it's own loop

* Fix race condition in provisioner jobs

* Fix requested changes
2022-03-28 19:19:28 -05:00
Kyle CarberryandGarrett 13cef7d07c feat: Support caching provisioner assets (#574)
* feat: Add AWS instance identity authentication

This allows zero-trust authentication for all AWS instances.

Prior to this, AWS instances could be used by passing `CODER_TOKEN`
as an environment variable to the startup script. AWS explicitly
states that secrets should not be passed in startup scripts because
it's user-readable.

* feat: Support caching provisioner assets

This caches the Terraform binary, and Terraform plugins.
Eventually, it could cache other temporary files.

* chore: fix linter

Co-authored-by: Garrett <garrett@coder.com>
2022-03-28 14:57:19 -05:00
Kyle Carberry b33dec9d38 feat: Add stage to build logs (#577)
* feat: Add stage to build logs

This adds a stage property to logs, and refactors the job logs
cliui.

It also adds tests to the cliui for build logs!

* Fix comments
2022-03-28 18:43:22 +00:00
Kyle Carberry 21fdb80825 fix: Parsing dynamic values for agent results in error (#564)
The logic required a constant value before, which disallowed dynamic
value injection into the agent. This isn't an accurate limitation,
so inverting the logic resolves it.
2022-03-25 17:21:39 +00:00
Kyle Carberry a06821c103 feat: Update Coder Terraform Provider to v0.2.1 (#563)
This update exposes the workspace name and owner, and changes
authentication methods to be explicit. Implicit authentication
added unnecessary complexity and introduced inconsistency.
2022-03-25 16:34:45 +00:00
Kyle Carberry c451f4e685 feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00