mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
docs: restructure docs (#14421)
Closes #13434 Supersedes #14182 --------- Co-authored-by: Ethan <39577870+ethanndickson@users.noreply.github.com> Co-authored-by: Ethan Dickson <ethan@coder.com> Co-authored-by: Ben Potter <ben@coder.com> Co-authored-by: Stephen Kirby <58410745+stirby@users.noreply.github.com> Co-authored-by: Stephen Kirby <me@skirby.dev> Co-authored-by: EdwardAngert <17991901+EdwardAngert@users.noreply.github.com> Co-authored-by: Edward Angert <EdwardAngert@users.noreply.github.com>
This commit is contained in:
Muhammad Atif Ali
co-authored by
Ethan
Ethan Dickson
Ben Potter
Stephen Kirby
Stephen Kirby
EdwardAngert
Edward Angert
Ethan
Ethan Dickson
Ben Potter
Stephen Kirby
Stephen Kirby
EdwardAngert
Edward Angert
parent
288df75686
commit
419eba5fb6
@@ -0,0 +1,344 @@
|
||||
# Deployment Health
|
||||
|
||||
Coder includes an operator-friendly deployment health page that provides a
|
||||
number of details about the health of your Coder deployment.
|
||||
|
||||
|
||||
|
||||
You can view it at `https://${CODER_URL}/health`, or you can alternatively view
|
||||
the
|
||||
[JSON response directly](../../reference/api/debug.md#debug-info-deployment-health).
|
||||
|
||||
The deployment health page is broken up into the following sections:
|
||||
|
||||
## Access URL
|
||||
|
||||
The Access URL section shows checks related to Coder's
|
||||
[access URL](../setup/index.md#access-url).
|
||||
|
||||
Coder will periodically send a GET request to `${CODER_ACCESS_URL}/healthz` and
|
||||
validate that the response is `200 OK`. The expected response body is also the
|
||||
string `OK`.
|
||||
|
||||
If there is an issue, you may see one of the following errors reported:
|
||||
|
||||
### EACS01
|
||||
|
||||
_Access URL not set_
|
||||
|
||||
**Problem:** no access URL has been configured.
|
||||
|
||||
**Solution:** configure an [access URL](../setup/index.md#access-url) for Coder.
|
||||
|
||||
### EACS02
|
||||
|
||||
_Access URL invalid_
|
||||
|
||||
**Problem:** `${CODER_ACCESS_URL}/healthz` is not a valid URL.
|
||||
|
||||
**Solution:** Ensure that the access URL is a valid URL accepted by
|
||||
[`url.Parse`](https://pkg.go.dev/net/url#Parse). Example:
|
||||
`https://dev.coder.com/`.
|
||||
|
||||
> **Tip:** You can check this [here](https://go.dev/play/p/CabcJZyTwt9).
|
||||
|
||||
### EACS03
|
||||
|
||||
_Failed to fetch `/healthz`_
|
||||
|
||||
**Problem:** Coder was unable to execute a GET request to
|
||||
`${CODER_ACCESS_URL}/healthz`.
|
||||
|
||||
This could be due to a number of reasons, including but not limited to:
|
||||
|
||||
- DNS lookup failure
|
||||
- A misconfigured firewall
|
||||
- A misconfigured reverse proxy
|
||||
- Invalid or expired SSL certificates
|
||||
|
||||
**Solution:** Investigate and resolve the root cause of the connection issue.
|
||||
|
||||
To troubleshoot further, you can log into the machine running Coder and attempt
|
||||
to run the following command:
|
||||
|
||||
```shell
|
||||
curl -v ${CODER_ACCESS_URL}/healthz
|
||||
# Expected output:
|
||||
# * Trying XXX.XXX.XXX.XXX:443
|
||||
# * Connected to https://coder.company.com (XXX.XXX.XXX.XXX) port 443 (#0)
|
||||
# [...]
|
||||
# OK
|
||||
```
|
||||
|
||||
The output of this command should aid further diagnosis.
|
||||
|
||||
### EACS04
|
||||
|
||||
_/healthz did not return 200 OK_
|
||||
|
||||
**Problem:** Coder was able to execute a GET request to
|
||||
`${CODER_ACCESS_URL}/healthz`, but the response code was not `200 OK` as
|
||||
expected.
|
||||
|
||||
This could mean, for instance, that:
|
||||
|
||||
- The request did not actually hit your Coder instance (potentially an incorrect
|
||||
DNS entry)
|
||||
- The request hit your Coder instance, but on an unexpected path (potentially a
|
||||
misconfigured reverse proxy)
|
||||
|
||||
**Solution:** Inspect the `HealthzResponse` in the health check output. This
|
||||
should give you a good indication of the root cause.
|
||||
|
||||
## Database
|
||||
|
||||
Coder continuously executes a short database query to validate that it can reach
|
||||
its configured database, and also measures the median latency over 5 attempts.
|
||||
|
||||
### EDB01
|
||||
|
||||
_Database Ping Failed_
|
||||
|
||||
**Problem:** This error code is returned if any attempt to execute this database
|
||||
query fails.
|
||||
|
||||
**Solution:** Investigate the health of the database.
|
||||
|
||||
### EDB02
|
||||
|
||||
_Database Latency High_
|
||||
|
||||
**Problem:** This code is returned if the median latency is higher than the
|
||||
[configured threshold](../../reference/cli/server.md#--health-check-threshold-database).
|
||||
This may not be an error as such, but is an indication of a potential issue.
|
||||
|
||||
**Solution:** Investigate the sizing of the configured database with regard to
|
||||
Coder's current activity and usage. It may be necessary to increase the
|
||||
resources allocated to Coder's database. Alternatively, you can raise the
|
||||
configured threshold to a higher value (this will not address the root cause).
|
||||
|
||||
> [!TIP]
|
||||
>
|
||||
> - You can enable
|
||||
> [detailed database metrics](../../reference/cli/server.md#--prometheus-collect-db-metrics)
|
||||
> in Coder's Prometheus endpoint.
|
||||
> - If you have [tracing enabled](../../reference/cli/server.md#--trace), these
|
||||
> traces may also contain useful information regarding Coder's database
|
||||
> activity.
|
||||
|
||||
## DERP
|
||||
|
||||
Coder workspace agents may use
|
||||
[DERP (Designated Encrypted Relay for Packets)](https://tailscale.com/blog/how-tailscale-works/#encrypted-tcp-relays-derp)
|
||||
to communicate with Coder. This requires connectivity to a number of configured
|
||||
[DERP servers](../../reference/cli/server.md#--derp-config-path) which are used
|
||||
to relay traffic between Coder and workspace agents. Coder periodically queries
|
||||
the health of its configured DERP servers and may return one or more of the
|
||||
following:
|
||||
|
||||
### EDERP01
|
||||
|
||||
_DERP Node Uses Websocket_
|
||||
|
||||
**Problem:** When Coder attempts to establish a connection to one or more DERP
|
||||
servers, it sends a specific `Upgrade: derp` HTTP header. Some load balancers
|
||||
may block this header, in which case Coder will fall back to
|
||||
`Upgrade: websocket`.
|
||||
|
||||
This is not necessarily a fatal error, but a possible indication of a
|
||||
misconfigured reverse HTTP proxy. Additionally, while workspace users should
|
||||
still be able to reach their workspaces, connection performance may be degraded.
|
||||
|
||||
> **Note:** This may also be shown if you have
|
||||
> [forced websocket connections for DERP](../../reference/cli/server.md#--derp-force-websockets).
|
||||
|
||||
**Solution:** ensure that any proxies you use allow connection upgrade with the
|
||||
`Upgrade: derp` header.
|
||||
|
||||
### EDERP02
|
||||
|
||||
_One or more DERP nodes are unhealthy_
|
||||
|
||||
**Problem:** This is shown if Coder is unable to reach one or more configured
|
||||
DERP servers. Clients will fall back to use the remaining DERP servers, but
|
||||
performance may be impacted for clients closest to the unhealthy DERP server.
|
||||
|
||||
**Solution:** Ensure that the DERP server is available and reachable over the
|
||||
network, for example:
|
||||
|
||||
```shell
|
||||
curl -v "https://coder.company.com/derp"
|
||||
# Expected output:
|
||||
# * Trying XXX.XXX.XXX.XXX
|
||||
# * Connected to https://coder.company.com (XXX.XXX.XXX.XXX) port 443 (#0)
|
||||
# DERP requires connection upgrade
|
||||
```
|
||||
|
||||
### ESTUN01
|
||||
|
||||
_No STUN servers available._
|
||||
|
||||
**Problem:** This is shown if no STUN servers are available. Coder will use STUN
|
||||
to establish [direct connections](../networking/stun.md). Without at least one
|
||||
working STUN server, direct connections may not be possible.
|
||||
|
||||
**Solution:** Ensure that the
|
||||
[configured STUN severs](../../reference/cli/server.md#--derp-server-stun-addresses)
|
||||
are reachable from Coder and that UDP traffic can be sent/received on the
|
||||
configured port.
|
||||
|
||||
### ESTUN02
|
||||
|
||||
_STUN returned different addresses; you may be behind a hard NAT._
|
||||
|
||||
**Problem:** This is a warning shown when multiple attempts to determine our
|
||||
public IP address/port via STUN resulted in different `ip:port` combinations.
|
||||
This is a sign that you are behind a "hard NAT", and may result in difficulty
|
||||
establishing direct connections. However, it does not mean that direct
|
||||
connections are impossible.
|
||||
|
||||
**Solution:** Engage with your network administrator.
|
||||
|
||||
## Websocket
|
||||
|
||||
Coder makes heavy use of [WebSockets](https://datatracker.ietf.org/doc/rfc6455/)
|
||||
for long-lived connections:
|
||||
|
||||
- Between users interacting with Coder's Web UI (for example, the built-in
|
||||
terminal, or VSCode Web),
|
||||
- Between workspace agents and `coderd`,
|
||||
- Between Coder [workspace proxies](../networking/workspace-proxies.md) and
|
||||
`coderd`.
|
||||
|
||||
Any issues causing failures to establish WebSocket connections will result in
|
||||
**severe** impairment of functionality for users. To validate this
|
||||
functionality, Coder will periodically attempt to establish a WebSocket
|
||||
connection with itself using the configured [Access URL](#access-url), send a
|
||||
message over the connection, and attempt to read back that same message.
|
||||
|
||||
### EWS01
|
||||
|
||||
_Failed to establish a WebSocket connection_
|
||||
|
||||
**Problem:** Coder was unable to establish a WebSocket connection over its own
|
||||
Access URL.
|
||||
|
||||
**Solution:** There are multiple possible causes of this problem:
|
||||
|
||||
1. Ensure that Coder's configured Access URL can be reached from the server
|
||||
running Coder, using standard troubleshooting tools like `curl`:
|
||||
|
||||
```shell
|
||||
curl -v "https://coder.company.com"
|
||||
```
|
||||
|
||||
2. Ensure that any reverse proxy that is serving Coder's configured access URL
|
||||
allows connection upgrade with the header `Upgrade: websocket`.
|
||||
|
||||
### EWS02
|
||||
|
||||
_Failed to echo a WebSocket message_
|
||||
|
||||
**Problem:** Coder was able to establish a WebSocket connection, but was unable
|
||||
to write a message.
|
||||
|
||||
**Solution:** There are multiple possible causes of this problem:
|
||||
|
||||
1. Validate that any reverse proxy servers in front of Coder's configured access
|
||||
URL are not prematurely closing the connection.
|
||||
2. Validate that the network link between Coder and the workspace proxy is
|
||||
stable, e.g. by using `ping`.
|
||||
3. Validate that any internal network infrastructure (for example, firewalls,
|
||||
proxies, VPNs) do not interfere with WebSocket connections.
|
||||
|
||||
## Workspace Proxy
|
||||
|
||||
If you have configured [Workspace Proxies](../networking/workspace-proxies.md),
|
||||
Coder will periodically query their availability and show their status here.
|
||||
|
||||
### EWP01
|
||||
|
||||
_Error Updating Workspace Proxy Health_
|
||||
|
||||
**Problem:** Coder was unable to query the connected workspace proxies for their
|
||||
health status.
|
||||
|
||||
**Solution:** This may be a transient issue. If it persists, it could signify a
|
||||
connectivity issue.
|
||||
|
||||
### EWP02
|
||||
|
||||
_Error Fetching Workspace Proxies_
|
||||
|
||||
**Problem:** Coder was unable to fetch the stored workspace proxy health data
|
||||
from the database.
|
||||
|
||||
**Solution:** This may be a transient issue. If it persists, it could signify an
|
||||
issue with Coder's configured database.
|
||||
|
||||
### EWP04
|
||||
|
||||
_One or more Workspace Proxies Unhealthy_
|
||||
|
||||
**Problem:** One or more workspace proxies are not reachable.
|
||||
|
||||
**Solution:** Ensure that Coder can establish a connection to the configured
|
||||
workspace proxies.
|
||||
|
||||
### EPD01
|
||||
|
||||
_No Provisioner Daemons Available_
|
||||
|
||||
**Problem:** No provisioner daemons are registered with Coder. No workspaces can
|
||||
be built until there is at least one provisioner daemon running.
|
||||
|
||||
**Solution:**
|
||||
|
||||
If you are using
|
||||
[External Provisioner Daemons](../provisioners.md#external-provisioners), ensure
|
||||
that they are able to successfully connect to Coder. Otherwise, ensure
|
||||
[`--provisioner-daemons`](../../reference/cli/server.md#--provisioner-daemons)
|
||||
is set to a value greater than 0.
|
||||
|
||||
> Note: This may be a transient issue if you are currently in the process of
|
||||
> updating your deployment.
|
||||
|
||||
### EPD02
|
||||
|
||||
_Provisioner Daemon Version Mismatch_
|
||||
|
||||
**Problem:** One or more provisioner daemons are more than one major or minor
|
||||
version out of date with the main deployment. It is important that provisioner
|
||||
daemons are updated at the same time as the main deployment to minimize the risk
|
||||
of API incompatibility.
|
||||
|
||||
**Solution:** Update the provisioner daemon to match the currently running
|
||||
version of Coder.
|
||||
|
||||
> Note: This may be a transient issue if you are currently in the process of
|
||||
> updating your deployment.
|
||||
|
||||
### EPD03
|
||||
|
||||
_Provisioner Daemon API Version Mismatch_
|
||||
|
||||
**Problem:** One or more provisioner daemons are using APIs that are marked as
|
||||
deprecated. These deprecated APIs may be removed in a future release of Coder,
|
||||
at which point the affected provisioner daemons will no longer be able to
|
||||
connect to Coder.
|
||||
|
||||
**Solution:** Update the provisioner daemon to match the currently running
|
||||
version of Coder.
|
||||
|
||||
> Note: This may be a transient issue if you are currently in the process of
|
||||
> updating your deployment.
|
||||
|
||||
## EUNKNOWN
|
||||
|
||||
_Unknown Error_
|
||||
|
||||
**Problem:** This error is shown when an unexpected error occurred evaluating
|
||||
deployment health. It may resolve on its own.
|
||||
|
||||
**Solution:** This may be a bug.
|
||||
[File a GitHub issue](https://github.com/coder/coder/issues/new)!
|
||||
@@ -0,0 +1,24 @@
|
||||
# Monitoring Coder
|
||||
|
||||
Learn about our the tools, techniques, and best practices to monitor Coder your
|
||||
Coder deployment.
|
||||
|
||||
## Quick Start: Observability Helm Chart
|
||||
|
||||
Deploy Prometheus, Grafana, Alert Manager, and pre-built dashboards on your
|
||||
Kubernetes cluster to monitor the Coder control plane, provisioners, and
|
||||
workspaces.
|
||||
|
||||
|
||||
|
||||
Learn how to install & read the docs on the
|
||||
[Observability Helm Chart GitHub](https://github.com/coder/observability)
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Logs](./logs.md): Learn how to access to Coder server logs, agent logs, and
|
||||
even how to expose Kubernetes pod scheduling logs.
|
||||
- [Metrics](./metrics.md): Learn about the valuable metrics to measure on a
|
||||
Coder deployment, regardless of your monitoring stack.
|
||||
- [Health Check](./health-check.md): Learn about the periodic health check and
|
||||
error codes that run on Coder deployments.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Logs
|
||||
|
||||
All Coder services log to standard output, which can be critical for identifying
|
||||
errors and monitoring Coder's deployment health. Like any service, logs can be
|
||||
captured via Splunk, Datadog, Grafana Loki, or other ingestion tools.
|
||||
|
||||
## `coderd` Logs
|
||||
|
||||
By default, the Coder server exports human-readable logs to standard output. You
|
||||
can access these logs via `kubectl logs deployment/coder -n <coder-namespace>`
|
||||
on Kubernetes or `journalctl -u coder` if you deployed Coder on a host
|
||||
machine/VM.
|
||||
|
||||
- To change the log format/location, you can set
|
||||
[`CODER_LOGGING_HUMAN`](../../reference/cli/server.md#--log-human) and
|
||||
[`CODER_LOGGING_JSON](../../reference/cli/server.md#--log-json) server config.
|
||||
options.
|
||||
- To only display certain types of logs, use
|
||||
the[`CODER_LOG_FILTER`](../../reference/cli/server.md#-l---log-filter) server
|
||||
config.
|
||||
|
||||
Events such as server errors, audit logs, user activities, and SSO & OpenID
|
||||
Connect logs are all captured in the `coderd` logs.
|
||||
|
||||
## `provisionerd` Logs
|
||||
|
||||
Logs for [external provisioners](../provisioners.md) are structured
|
||||
[and configured](../../reference/cli/provisioner_start.md#--log-human) similarly
|
||||
to `coderd` logs. Use these logs to troubleshoot and monitor the Terraform
|
||||
operations behind workspaces and templates.
|
||||
|
||||
## Workspace Logs
|
||||
|
||||
The [Coder agent](../infrastructure/architecture.md#agents) inside workspaces
|
||||
provides useful logs around workspace-to-server and client-to-workspace
|
||||
connections. For Kubernetes workspaces, these are typically the pod logs as the
|
||||
agent runs via the container entrypoint.
|
||||
|
||||
Agent logs are also stored in the workspace filesystem by default:
|
||||
|
||||
- macOS/Linux: `/tmp/coder-agent.log`
|
||||
- Windows: Refer to the template code (e.g.
|
||||
[azure-windows](https://github.com/coder/coder/blob/2cfadad023cb7f4f85710cff0b21ac46bdb5a845/examples/templates/azure-windows/Initialize.ps1.tftpl#L64))
|
||||
to see where logs are stored.
|
||||
|
||||
> Note: Logs are truncated once they reach 5MB in size.
|
||||
|
||||
Startup script logs are also stored in the temporary directory of macOS and
|
||||
Linux workspaces.
|
||||
|
||||
## Kubernetes Event Logs
|
||||
|
||||
Sometimes, a workspace may take a while to start or even fail to start due to
|
||||
underlying events on the Kubernetes cluster such as a node being out of
|
||||
resources or a missing image. You can install
|
||||
[coder-logstream-kube](../integrations/kubernetes-logs.md) to stream Kubernetes
|
||||
events to the Coder UI.
|
||||
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
# Deployment Metrics
|
||||
|
||||
Coder exposes many metrics which give insight into the current state of a live
|
||||
Coder deployment. Our metrics are designed to be consumed by a
|
||||
[Prometheus server](https://prometheus.io/).
|
||||
|
||||
If you don't have an Prometheus server installed, you can follow the Prometheus
|
||||
[Getting started](https://prometheus.io/docs/prometheus/latest/getting_started/)
|
||||
guide.
|
||||
|
||||
### Setting up metrics
|
||||
|
||||
To set up metrics monitoring, please read our
|
||||
[Prometheus integration guide](../integrations/prometheus.md). The following
|
||||
links point to relevant sections there.
|
||||
|
||||
- [Enable Prometheus metrics](../integrations/prometheus.md#enable-prometheus-metrics)
|
||||
in the control plane
|
||||
- [Enable the Prometheus endpoint in Helm](../integrations/prometheus.md#kubernetes-deployment)
|
||||
(Kubernetes users only)
|
||||
- [Configure Prometheus to scrape Coder metrics](../integrations/prometheus.md#prometheus-configuration)
|
||||
- [See the list of available metrics](../integrations/prometheus.md#available-metrics)
|
||||
@@ -0,0 +1,302 @@
|
||||
# Notifications
|
||||
|
||||
Notifications are sent by Coder in response to specific internal events, such as
|
||||
a workspace being deleted or a user being created.
|
||||
|
||||
## Enable experiment
|
||||
|
||||
In order to activate the notifications feature on Coder v2.15.X, you'll need to
|
||||
enable the `notifications` experiment. Notifications are enabled by default
|
||||
starting in v2.16.0.
|
||||
|
||||
```bash
|
||||
# Using the CLI flag
|
||||
$ coder server --experiments=notifications
|
||||
|
||||
# Alternatively, using the `CODER_EXPERIMENTS` environment variable
|
||||
$ CODER_EXPERIMENTS=notifications coder server
|
||||
```
|
||||
|
||||
More information on experiments can be found
|
||||
[here](https://coder.com/docs/contributing/feature-stages#experimental-features).
|
||||
|
||||
## Event Types
|
||||
|
||||
Notifications are sent in response to internal events, to alert the affected
|
||||
user(s) of this event. Currently we support the following list of events:
|
||||
|
||||
### Workspace Events
|
||||
|
||||
_These notifications are sent to the workspace owner._
|
||||
|
||||
- Workspace Deleted
|
||||
- Workspace Manual Build Failure
|
||||
- Workspace Automatic Build Failure
|
||||
- Workspace Automatically Updated
|
||||
- Workspace Dormant
|
||||
- Workspace Marked For Deletion
|
||||
|
||||
### User Events
|
||||
|
||||
_These notifications are sent to users with **owner** and **user admin** roles._
|
||||
|
||||
- User Account Created
|
||||
- User Account Deleted
|
||||
- User Account Suspended
|
||||
- User Account Activated
|
||||
- _(coming soon) User Password Reset_
|
||||
- _(coming soon) User Email Verification_
|
||||
|
||||
_These notifications are sent to the user themselves._
|
||||
|
||||
- User Account Suspended
|
||||
- User Account Activated
|
||||
|
||||
### Template Events
|
||||
|
||||
_These notifications are sent to users with **template admin** roles._
|
||||
|
||||
- Template Deleted
|
||||
|
||||
## Configuration
|
||||
|
||||
You can modify the notification delivery behavior using the following server
|
||||
flags.
|
||||
|
||||
| Required | CLI | Env | Type | Description | Default |
|
||||
| :------: | ----------------------------------- | --------------------------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------- | ------- |
|
||||
| ✔️ | `--notifications-dispatch-timeout` | `CODER_NOTIFICATIONS_DISPATCH_TIMEOUT` | `duration` | How long to wait while a notification is being sent before giving up. | 1m |
|
||||
| ✔️ | `--notifications-method` | `CODER_NOTIFICATIONS_METHOD` | `string` | Which delivery method to use (available options: 'smtp', 'webhook'). See [Delivery Methods](#delivery-methods) below. | smtp |
|
||||
| -️ | `--notifications-max-send-attempts` | `CODER_NOTIFICATIONS_MAX_SEND_ATTEMPTS` | `int` | The upper limit of attempts to send a notification. | 5 |
|
||||
|
||||
## Delivery Methods
|
||||
|
||||
Notifications can currently be delivered by either SMTP or webhook. Each message
|
||||
can only be delivered to one method, and this method is configured globally with
|
||||
[`CODER_NOTIFICATIONS_METHOD`](../../../reference/cli/server.md#--notifications-method)
|
||||
(default: `smtp`).
|
||||
|
||||
Enterprise customers can configure which method to use for each of the supported
|
||||
[Events](#events); see the [Preferences](#preferences) section below for more
|
||||
details.
|
||||
|
||||
## SMTP (Email)
|
||||
|
||||
Use the `smtp` method to deliver notifications by email to your users. Coder
|
||||
does not ship with an SMTP server, so you will need to configure Coder to use an
|
||||
existing one.
|
||||
|
||||
**Server Settings:**
|
||||
|
||||
| Required | CLI | Env | Type | Description | Default |
|
||||
| :------: | --------------------------------- | ------------------------------------- | ----------- | ----------------------------------------- | ------------- |
|
||||
| ✔️ | `--notifications-email-from` | `CODER_NOTIFICATIONS_EMAIL_FROM` | `string` | The sender's address to use. | |
|
||||
| ✔️ | `--notifications-email-smarthost` | `CODER_NOTIFICATIONS_EMAIL_SMARTHOST` | `host:port` | The SMTP relay to send messages through. | localhost:587 |
|
||||
| ✔️ | `--notifications-email-hello` | `CODER_NOTIFICATIONS_EMAIL_HELLO` | `string` | The hostname identifying the SMTP server. | localhost |
|
||||
|
||||
**Authentication Settings:**
|
||||
|
||||
| Required | CLI | Env | Type | Description |
|
||||
| :------: | ------------------------------------------ | ---------------------------------------------- | -------- | ------------------------------------------------------------------------- |
|
||||
| - | `--notifications-email-auth-username` | `CODER_NOTIFICATIONS_EMAIL_AUTH_USERNAME` | `string` | Username to use with PLAIN/LOGIN authentication. |
|
||||
| - | `--notifications-email-auth-password` | `CODER_NOTIFICATIONS_EMAIL_AUTH_PASSWORD` | `string` | Password to use with PLAIN/LOGIN authentication. |
|
||||
| - | `--notifications-email-auth-password-file` | `CODER_NOTIFICATIONS_EMAIL_AUTH_PASSWORD_FILE` | `string` | File from which to load password for use with PLAIN/LOGIN authentication. |
|
||||
| - | `--notifications-email-auth-identity` | `CODER_NOTIFICATIONS_EMAIL_AUTH_IDENTITY` | `string` | Identity to use with PLAIN authentication. |
|
||||
|
||||
**TLS Settings:**
|
||||
|
||||
| Required | CLI | Env | Type | Description | Default |
|
||||
| :------: | ----------------------------------------- | ------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
|
||||
| - | `--notifications-email-force-tls` | `CODER_NOTIFICATIONS_EMAIL_FORCE_TLS` | `bool` | Force a TLS connection to the configured SMTP smarthost. If port 465 is used, TLS will be forced. See https://datatracker.ietf.org/doc/html/rfc8314#section-3.3. | false |
|
||||
| - | `--notifications-email-tls-starttls` | `CODER_NOTIFICATIONS_EMAIL_TLS_STARTTLS` | `bool` | Enable STARTTLS to upgrade insecure SMTP connections using TLS. Ignored if `CODER_NOTIFICATIONS_EMAIL_FORCE_TLS` is set. | false |
|
||||
| - | `--notifications-email-tls-skip-verify` | `CODER_NOTIFICATIONS_EMAIL_TLS_SKIPVERIFY` | `bool` | Skip verification of the target server's certificate (**insecure**). | false |
|
||||
| - | `--notifications-email-tls-server-name` | `CODER_NOTIFICATIONS_EMAIL_TLS_SERVERNAME` | `string` | Server name to verify against the target certificate. | |
|
||||
| - | `--notifications-email-tls-cert-file` | `CODER_NOTIFICATIONS_EMAIL_TLS_CERTFILE` | `string` | Certificate file to use. | |
|
||||
| - | `--notifications-email-tls-cert-key-file` | `CODER_NOTIFICATIONS_EMAIL_TLS_CERTKEYFILE` | `string` | Certificate key file to use. | |
|
||||
|
||||
**NOTE:** you _MUST_ use `CODER_NOTIFICATIONS_EMAIL_FORCE_TLS` if your smarthost
|
||||
supports TLS on a port other than `465`.
|
||||
|
||||
### Send emails using G-Suite
|
||||
|
||||
After setting the required fields above:
|
||||
|
||||
1. Create an [App Password](https://myaccount.google.com/apppasswords) using the
|
||||
account you wish to send from
|
||||
2. Set the following configuration options:
|
||||
```
|
||||
CODER_NOTIFICATIONS_EMAIL_SMARTHOST=smtp.gmail.com:465
|
||||
CODER_NOTIFICATIONS_EMAIL_AUTH_USERNAME=<user>@<domain>
|
||||
CODER_NOTIFICATIONS_EMAIL_AUTH_PASSWORD="<app password created above>"
|
||||
```
|
||||
|
||||
See
|
||||
[this help article from Google](https://support.google.com/a/answer/176600?hl=en)
|
||||
for more options.
|
||||
|
||||
### Send emails using Outlook.com
|
||||
|
||||
After setting the required fields above:
|
||||
|
||||
1. Setup an account on Microsoft 365 or outlook.com
|
||||
2. Set the following configuration options:
|
||||
```
|
||||
CODER_NOTIFICATIONS_EMAIL_SMARTHOST=smtp-mail.outlook.com:587
|
||||
CODER_NOTIFICATIONS_EMAIL_TLS_STARTTLS=true
|
||||
CODER_NOTIFICATIONS_EMAIL_AUTH_USERNAME=<user>@<domain>
|
||||
CODER_NOTIFICATIONS_EMAIL_AUTH_PASSWORD="<account password>"
|
||||
```
|
||||
|
||||
See
|
||||
[this help article from Microsoft](https://support.microsoft.com/en-us/office/pop-imap-and-smtp-settings-for-outlook-com-d088b986-291d-42b8-9564-9c414e2aa040)
|
||||
for more options.
|
||||
|
||||
## Webhook
|
||||
|
||||
The webhook delivery method sends an HTTP POST request to the defined endpoint.
|
||||
The purpose of webhook notifications is to enable integrations with other
|
||||
systems.
|
||||
|
||||
**Settings**:
|
||||
|
||||
| Required | CLI | Env | Type | Description |
|
||||
| :------: | ---------------------------------- | -------------------------------------- | ----- | --------------------------------------- |
|
||||
| ✔️ | `--notifications-webhook-endpoint` | `CODER_NOTIFICATIONS_WEBHOOK_ENDPOINT` | `url` | The endpoint to which to send webhooks. |
|
||||
|
||||
Here is an example payload for Coder's webhook notification:
|
||||
|
||||
```json
|
||||
{
|
||||
"_version": "1.0",
|
||||
"msg_id": "88750cad-77d4-4663-8bc0-f46855f5019b",
|
||||
"payload": {
|
||||
"_version": "1.0",
|
||||
"notification_name": "Workspace Deleted",
|
||||
"user_id": "4ac34fcb-8155-44d5-8301-e3cd46e88b35",
|
||||
"user_email": "danny@coder.com",
|
||||
"user_name": "danny",
|
||||
"user_username": "danny",
|
||||
"actions": [
|
||||
{
|
||||
"label": "View workspaces",
|
||||
"url": "https://et23ntkhpueak.pit-1.try.coder.app/workspaces"
|
||||
},
|
||||
{
|
||||
"label": "View templates",
|
||||
"url": "https://et23ntkhpueak.pit-1.try.coder.app/templates"
|
||||
}
|
||||
],
|
||||
"labels": {
|
||||
"initiator": "danny",
|
||||
"name": "my-workspace",
|
||||
"reason": "initiated by user"
|
||||
}
|
||||
},
|
||||
"title": "Workspace \"my-workspace\" deleted",
|
||||
"body": "Hi danny\n\nYour workspace my-workspace was deleted.\nThe specified reason was \"initiated by user (danny)\"."
|
||||
}
|
||||
```
|
||||
|
||||
The top-level object has these keys:
|
||||
|
||||
- `_version`: describes the version of this schema; follows semantic versioning
|
||||
- `msg_id`: the UUID of the notification (matches the ID in the
|
||||
`notification_messages` table)
|
||||
- `payload`: contains the specific details of the notification; described below
|
||||
- `title`: the title of the notification message (equivalent to a subject in
|
||||
SMTP delivery)
|
||||
- `body`: the body of the notification message (equivalent to the message body
|
||||
in SMTP delivery)
|
||||
|
||||
The `payload` object has these keys:
|
||||
|
||||
- `_version`: describes the version of this inner schema; follows semantic
|
||||
versioning
|
||||
- `notification_name`: name of the event which triggered the notification
|
||||
- `user_id`: Coder internal user identifier of the target user (UUID)
|
||||
- `user_email`: email address of the target user
|
||||
- `user_name`: name of the target user
|
||||
- `user_username`: username of the target user
|
||||
- `actions`: a list of CTAs (Call-To-Action); these are mainly relevant for SMTP
|
||||
delivery in which they're shown as buttons
|
||||
- `labels`: dynamic map of zero or more string key-value pairs; these vary from
|
||||
event to event
|
||||
|
||||
## User Preferences
|
||||
|
||||
All users have the option to opt-out of any notifications. Go to **Account** ->
|
||||
**Notifications** to turn notifications on or off. The delivery method for each
|
||||
notification is indicated on the right hand side of this table.
|
||||
|
||||
|
||||
|
||||
## Delivery Preferences (enterprise) (premium)
|
||||
|
||||
Administrators can configure which delivery methods are used for each different
|
||||
[event type](#event-types).
|
||||
|
||||
|
||||
|
||||
You can find this page under
|
||||
`https://$CODER_ACCESS_URL/deployment/notifications?tab=events`.
|
||||
|
||||
## Stop sending notifications
|
||||
|
||||
Administrators may wish to stop _all_ notifications across the deployment. We
|
||||
support a killswitch in the CLI for these cases.
|
||||
|
||||
To pause sending notifications, execute
|
||||
[`coder notifications pause`](../../../reference/cli/notifications_pause.md).
|
||||
|
||||
To resume sending notifications, execute
|
||||
[`coder notifications resume`](../../../reference/cli/notifications_resume.md).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If notifications are not being delivered, use the following methods to
|
||||
troubleshoot:
|
||||
|
||||
1. Ensure notifications are being added to the `notification_messages` table
|
||||
2. Review any error messages in the `status_reason` column, should an error have
|
||||
occurred
|
||||
3. Review the logs (search for the term `notifications`) for diagnostic
|
||||
information<br> _If you do not see any relevant logs, set
|
||||
`CODER_VERBOSE=true` or `--verbose` to output debug logs_
|
||||
|
||||
## Internals
|
||||
|
||||
The notification system is built to operate concurrently in a single- or
|
||||
multi-replica Coder deployment, and has a built-in retry mechanism. It uses the
|
||||
configured Postgres database to store notifications in a queue and facilitate
|
||||
concurrency.
|
||||
|
||||
All messages are stored in the `notification_messages` table.
|
||||
|
||||
Messages older than 7 days are deleted.
|
||||
|
||||
### Message States
|
||||
|
||||
|
||||
|
||||
_A notifier here refers to a Coder replica which is responsible for dispatching
|
||||
the notification. All running replicas act as notifiers to process pending
|
||||
messages._
|
||||
|
||||
- a message begins in `pending` state
|
||||
- transitions to `leased` when a Coder replica acquires new messages from the
|
||||
database
|
||||
- new messages are checked for every `CODER_NOTIFICATIONS_FETCH_INTERVAL`
|
||||
(default: 15s)
|
||||
- if a message is delivered successfully, it transitions to `sent` state
|
||||
- if a message encounters a non-retryable error (e.g. misconfiguration), it
|
||||
transitions to `permanent_failure`
|
||||
- if a message encounters a retryable error (e.g. temporary server outage), it
|
||||
transitions to `temporary_failure`
|
||||
- this message will be retried up to `CODER_NOTIFICATIONS_MAX_SEND_ATTEMPTS`
|
||||
(default: 5)
|
||||
- this message will transition back to `pending` state after
|
||||
`CODER_NOTIFICATIONS_RETRY_INTERVAL` (default: 5m) and be retried
|
||||
- after `CODER_NOTIFICATIONS_MAX_SEND_ATTEMPTS` is exceeded, it transitions to
|
||||
`permanent_failure`
|
||||
|
||||
See [Troubleshooting](#troubleshooting) above for more details.
|
||||
@@ -0,0 +1,206 @@
|
||||
# Slack Notifications
|
||||
|
||||
[Slack](https://slack.com/) is a popular messaging platform designed for teams
|
||||
and businesses, enabling real-time collaboration through channels, direct
|
||||
messages, and integrations with external tools. With Coder's integration, you
|
||||
can enable automated notifications directly within a self-hosted
|
||||
[Slack app](https://api.slack.com/apps), keeping your team updated on key events
|
||||
in your Coder environment.
|
||||
|
||||
Administrators can configure Coder to send notifications via an incoming webhook
|
||||
endpoint. These notifications will be delivered as Slack messages direct to the
|
||||
user. Routing is based on the user's email address, and this should be
|
||||
consistent between Slack and their Coder login.
|
||||
|
||||
## Requirements
|
||||
|
||||
Before setting up Slack notifications, ensure that you have the following:
|
||||
|
||||
- Administrator access to the Slack platform to create apps
|
||||
- Coder platform v2.15.0 or greater with
|
||||
[notifications enabled](./index.md#enable-experiment) for versions <v2.16.0
|
||||
|
||||
## Create Slack Application
|
||||
|
||||
To integrate Slack with Coder, follow these steps to create a Slack application:
|
||||
|
||||
1. Go to the [Slack Apps](https://api.slack.com/apps) dashboard and create a new
|
||||
Slack App.
|
||||
|
||||
2. Under "Basic Information," you'll find a "Signing Secret." The Slack
|
||||
application uses it to
|
||||
[verify requests](https://api.slack.com/authentication/verifying-requests-from-slack)
|
||||
coming from Slack.
|
||||
|
||||
3. Under "OAuth & Permissions", add the following OAuth scopes:
|
||||
|
||||
- `chat:write`: To send messages as the app.
|
||||
- `users:read`: To find the user details.
|
||||
- `users:read.email`: To find user emails.
|
||||
|
||||
4. Install the app to your workspace and note down the **Bot User OAuth Token**
|
||||
from the "OAuth & Permissions" section.
|
||||
|
||||
## Build a Webserver to Receive Webhooks
|
||||
|
||||
The Slack bot for Coder runs as a _Bolt application_, which is a framework
|
||||
designed for building Slack apps using the Slack API.
|
||||
[Bolt for JavaScript](https://github.com/slackapi/bolt-js) provides an
|
||||
easy-to-use API for responding to events, commands, and interactions from Slack.
|
||||
|
||||
To build the server to receive webhooks and interact with Slack:
|
||||
|
||||
1. Initialize your project by running:
|
||||
|
||||
```bash
|
||||
npm init -y
|
||||
```
|
||||
|
||||
2. Install the Bolt library:
|
||||
|
||||
```bash
|
||||
npm install @slack/bolt
|
||||
```
|
||||
|
||||
3. Create and edit the `app.js` file. Below is an example of the basic
|
||||
structure:
|
||||
|
||||
```js
|
||||
const { App, LogLevel, ExpressReceiver } = require("@slack/bolt");
|
||||
const bodyParser = require("body-parser");
|
||||
|
||||
const port = process.env.PORT || 6000;
|
||||
|
||||
// Create a Bolt Receiver
|
||||
const receiver = new ExpressReceiver({
|
||||
signingSecret: process.env.SLACK_SIGNING_SECRET,
|
||||
});
|
||||
receiver.router.use(bodyParser.json());
|
||||
|
||||
// Create the Bolt App, using the receiver
|
||||
const app = new App({
|
||||
token: process.env.SLACK_BOT_TOKEN,
|
||||
logLevel: LogLevel.DEBUG,
|
||||
receiver,
|
||||
});
|
||||
|
||||
receiver.router.post("/v1/webhook", async (req, res) => {
|
||||
try {
|
||||
if (!req.body) {
|
||||
return res.status(400).send("Error: request body is missing");
|
||||
}
|
||||
|
||||
const { title, body } = req.body;
|
||||
if (!title || !body) {
|
||||
return res.status(400).send('Error: missing fields: "title", or "body"');
|
||||
}
|
||||
|
||||
const payload = req.body.payload;
|
||||
if (!payload) {
|
||||
return res.status(400).send('Error: missing "payload" field');
|
||||
}
|
||||
|
||||
const { user_email, actions } = payload;
|
||||
if (!user_email || !actions) {
|
||||
return res
|
||||
.status(400)
|
||||
.send('Error: missing fields: "user_email", "actions"');
|
||||
}
|
||||
|
||||
// Get the user ID using Slack API
|
||||
const userByEmail = await app.client.users.lookupByEmail({
|
||||
email: user_email,
|
||||
});
|
||||
|
||||
const slackMessage = {
|
||||
channel: userByEmail.user.id,
|
||||
text: body,
|
||||
blocks: [
|
||||
{
|
||||
type: "header",
|
||||
text: { type: "plain_text", text: title },
|
||||
},
|
||||
{
|
||||
type: "section",
|
||||
text: { type: "mrkdwn", text: body },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
// Add action buttons if they exist
|
||||
if (actions && actions.length > 0) {
|
||||
slackMessage.blocks.push({
|
||||
type: "actions",
|
||||
elements: actions.map((action) => ({
|
||||
type: "button",
|
||||
text: { type: "plain_text", text: action.label },
|
||||
url: action.url,
|
||||
})),
|
||||
});
|
||||
}
|
||||
|
||||
// Post message to the user on Slack
|
||||
await app.client.chat.postMessage(slackMessage);
|
||||
|
||||
res.status(204).send();
|
||||
} catch (error) {
|
||||
console.error("Error sending message:", error);
|
||||
res.status(500).send();
|
||||
}
|
||||
});
|
||||
|
||||
// Acknowledge clicks on link_button, otherwise Slack UI
|
||||
// complains about missing events.
|
||||
app.action("button_click", async ({ body, ack, say }) => {
|
||||
await ack(); // no specific action needed
|
||||
});
|
||||
|
||||
// Start the Bolt app
|
||||
(async () => {
|
||||
await app.start(port);
|
||||
console.log("⚡️ Coder Slack bot is running!");
|
||||
})();
|
||||
```
|
||||
|
||||
3. Set environment variables to identify the Slack app:
|
||||
|
||||
```bash
|
||||
export SLACK_BOT_TOKEN=xoxb-...
|
||||
export SLACK_SIGNING_SECRET=0da4b...
|
||||
```
|
||||
|
||||
4. Start the web application by running:
|
||||
|
||||
```bash
|
||||
node app.js
|
||||
```
|
||||
|
||||
## Enable Interactivity in Slack
|
||||
|
||||
Slack requires the bot to acknowledge when a user clicks on a URL action button.
|
||||
This is handled by setting up interactivity.
|
||||
|
||||
1. Under "Interactivity & Shortcuts" in your Slack app settings, set the Request
|
||||
URL to match the public URL of your web server's endpoint.
|
||||
|
||||
> Notice: You can use any public endpoint that accepts and responds to POST
|
||||
> requests with HTTP 200. For temporary testing, you can set it to
|
||||
> `https://httpbin.org/status/200`.
|
||||
|
||||
Once this is set, Slack will send interaction payloads to your server, which
|
||||
must respond appropriately.
|
||||
|
||||
## Enable Webhook Integration in Coder
|
||||
|
||||
To enable webhook integration in Coder, ensure the "notifications"
|
||||
[experiment is activated](./index.md#enable-experiment) (only required in
|
||||
v2.15.X).
|
||||
|
||||
Then, define the POST webhook endpoint matching the deployed Slack bot:
|
||||
|
||||
```bash
|
||||
export CODER_NOTIFICATIONS_WEBHOOK_ENDPOINT=http://localhost:6000/v1/webhook`
|
||||
```
|
||||
|
||||
Finally, go to the **Notification Settings** in Coder and switch the notifier to
|
||||
**Webhook**.
|
||||
@@ -0,0 +1,157 @@
|
||||
# Microsoft Teams Notifications
|
||||
|
||||
[Microsoft Teams](https://www.microsoft.com/en-us/microsoft-teams) is a widely
|
||||
used collaboration platform, and with Coder's integration, you can enable
|
||||
automated notifications directly within Teams using workflows and
|
||||
[Adaptive Cards](https://adaptivecards.io/)
|
||||
|
||||
Administrators can configure Coder to send notifications via an incoming webhook
|
||||
endpoint. These notifications appear as messages in Teams chats, either with the
|
||||
Flow Bot or a specified user/service account.
|
||||
|
||||
## Requirements
|
||||
|
||||
Before setting up Microsoft Teams notifications, ensure that you have the
|
||||
following:
|
||||
|
||||
- Administrator access to the Teams platform
|
||||
- Coder platform with [notifications enabled](./index.md#enable-experiment)
|
||||
|
||||
## Build Teams Workflow
|
||||
|
||||
The process of setting up a Teams workflow consists of three key steps:
|
||||
|
||||
1. Configure the Webhook Trigger.
|
||||
|
||||
Begin by configuring the trigger: **"When a Teams webhook request is
|
||||
received"**.
|
||||
|
||||
Ensure the trigger access level is set to **"Anyone"**.
|
||||
|
||||
2. Setup the JSON Parsing Action.
|
||||
|
||||
Next, add the **"Parse JSON"** action, linking the content to the **"Body"**
|
||||
of the received webhook request. Use the following schema to parse the
|
||||
notification payload:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"_version": {
|
||||
"type": "string"
|
||||
},
|
||||
"payload": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"_version": {
|
||||
"type": "string"
|
||||
},
|
||||
"user_email": {
|
||||
"type": "string"
|
||||
},
|
||||
"actions": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": "string"
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": ["label", "url"]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"body": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This action parses the notification's title, body, and the recipient's email
|
||||
address.
|
||||
|
||||
3. Configure the Adaptive Card Action.
|
||||
|
||||
Finally, set up the **"Post Adaptive Card in a chat or channel"** action
|
||||
with the following recommended settings:
|
||||
|
||||
**Post as**: Flow Bot
|
||||
|
||||
**Post in**: Chat with Flow Bot
|
||||
|
||||
**Recipient**: `user_email`
|
||||
|
||||
Use the following _Adaptive Card_ template:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://adaptivecards.io/schemas/adaptive-card.json",
|
||||
"type": "AdaptiveCard",
|
||||
"version": "1.0",
|
||||
"body": [
|
||||
{
|
||||
"type": "Image",
|
||||
"url": "https://coder.com/coder-logo-horizontal.png",
|
||||
"height": "40px",
|
||||
"altText": "Coder",
|
||||
"horizontalAlignment": "center"
|
||||
},
|
||||
{
|
||||
"type": "TextBlock",
|
||||
"text": "**@{replace(body('Parse_JSON')?['title'], '"', '\"')}**"
|
||||
},
|
||||
{
|
||||
"type": "TextBlock",
|
||||
"text": "@{replace(body('Parse_JSON')?['body'], '"', '\"')}",
|
||||
"wrap": true
|
||||
},
|
||||
{
|
||||
"type": "ActionSet",
|
||||
"actions": [@{replace(replace(join(body('Parse_JSON')?['payload']?['actions'], ','), '{', '{"type": "Action.OpenUrl",'), '"label"', '"title"')}]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
_Notice_: The Coder `actions` format differs from the `ActionSet` schema, so
|
||||
its properties need to be modified: include `Action.OpenUrl` type, rename
|
||||
`label` to `title`. Unfortunately, there is no straightforward solution for
|
||||
`for-each` pattern.
|
||||
|
||||
Feel free to customize the payload to modify the logo, notification title,
|
||||
or body content to suit your needs.
|
||||
|
||||
## Enable Webhook Integration
|
||||
|
||||
To enable webhook integration in Coder, ensure the "notifications"
|
||||
[experiment is activated](./index.md#enable-experiment) (only required in
|
||||
v2.15.X).
|
||||
|
||||
Then, define the POST webhook endpoint created by your Teams workflow:
|
||||
|
||||
```bash
|
||||
export CODER_NOTIFICATIONS_WEBHOOK_ENDPOINT=https://prod-16.eastus.logic.azure.com:443/workflows/f8fbe3e8211e4b638...`
|
||||
```
|
||||
|
||||
Finally, go to the **Notification Settings** in Coder and switch the notifier to
|
||||
**Webhook**.
|
||||
|
||||
## Limitations
|
||||
|
||||
1. **Public Webhook Trigger**: The Teams webhook trigger must be open to the
|
||||
public (**"Anyone"** can send the payload). It's recommended to keep the
|
||||
endpoint secret and apply additional authorization layers to protect against
|
||||
unauthorized access.
|
||||
|
||||
2. **Markdown Support in Adaptive Cards**: Note that Adaptive Cards support a
|
||||
[limited set of Markdown tags](https://learn.microsoft.com/en-us/microsoftteams/platform/task-modules-and-cards/cards/cards-format?tabs=adaptive-md%2Cdesktop%2Cconnector-html).
|
||||
Reference in New Issue
Block a user