opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-01 14:53:15 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs: add data retention documentation (#21038)

Browse Source 
Document configurable retention policies for Audit Logs, Connection Logs,
and API keys. Add new data-retention.md page and update existing docs to
reference it.

Depends on #21021
Updates #20743
This commit is contained in:
Mathias Fredriksson
2025-12-02 17:47:36 +02:00
committed by GitHub
parent 9ec90cf2e7
commit d9888ced11
5 changed files with 198 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin/monitoring/connection-logs.md
+8
View File
@@ -106,6 +106,14 @@ connection log entry, when `code-server` is opened:
[API] 2025-07-03 06:57:16.157 [info] coderd: connection_log request_id=de3f6004-6cc1-4880-a296-d7c6ca1abf75 ID=f0249951-d454-48f6-9504-e73340fa07b7 Time="2025-07-03T06:57:16.144719Z" OrganizationID=0665a54f-0b77-4a58-94aa-59646fa38a74 WorkspaceOwnerID=6dea5f8c-ecec-4cf0-a5bd-bc2c63af2efa WorkspaceID=3c0b37c8-e58c-4980-b9a1-2732410480a5 WorkspaceName=dev AgentName=main Type=workspace_app Code=200 Ip=127.0.0.1 UserAgent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36" UserID=6dea5f8c-ecec-4cf0-a5bd-bc2c63af2efa SlugOrPort=code-server ConnectionID=<nil> DisconnectReason="" ConnectionStatus=connected
```
## Data Retention
Coder supports configurable retention policies that automatically purge old
Connection Logs. To enable automated purging, configure the
`--connection-logs-retention` flag or `CODER_CONNECTION_LOGS_RETENTION`
environment variable. For comprehensive configuration options, see
[Data Retention](../setup/data-retention.md).
## How to Enable Connection Logs
This feature is only available with a [Premium license](../licensing/index.md).
docs/admin/monitoring/index.md
+1
View File
@@ -22,3 +22,4 @@ Learn how to install & read the docs on the
Coder deployment, regardless of your monitoring stack.
- [Health Check](./health-check.md): Learn about the periodic health check and
error codes that run on Coder deployments.
- [Connection Logs](./connection-logs.md): Monitor connections to workspaces.
docs/admin/security/audit-logs.md
+15 -2
View File
@@ -132,8 +132,21 @@ log entry:
> Audit Logs provide critical security and compliance information. Purging Audit Logs may impact your organization's ability
> to investigate security incidents or meet compliance requirements. Consult your security and compliance teams before purging any audit data.
Audit Logs are not automatically purged from the database, though they can account for a large amount of disk usage.
Use the following query to determine the amount of disk space used by the `audit_logs` table.
### Data Retention
Coder supports configurable retention policies that automatically purge old
Audit Logs. To enable automated purging, configure the
`--audit-logs-retention` flag or `CODER_AUDIT_LOGS_RETENTION` environment
variable. For comprehensive configuration options, see
[Data Retention](../setup/data-retention.md).
### Manual Purging
Alternatively, you can purge Audit Logs manually by running SQL queries
directly against the database.
Audit Logs can account for a large amount of disk usage. Use the following
query to determine the amount of disk space used by the `audit_logs` table.
```sql
SELECT
docs/admin/setup/data-retention.md
+169
View File
@@ -0,0 +1,169 @@
# Data Retention
Coder supports configurable retention policies that automatically purge old
Audit Logs, Connection Logs, and API keys. These policies help manage database
growth by removing records older than a specified duration.
## Overview
Large deployments can accumulate significant amounts of data over time.
Retention policies help you:
- **Reduce database size**: Automatically remove old records to free disk space.
- **Improve performance**: Smaller tables mean faster queries and backups.
- **Meet compliance requirements**: Configure retention periods that align with
your organization's data retention policies.
> [!NOTE]
> Retention policies are disabled by default (set to `0`) to preserve existing
> behavior. The only exception is API keys, which defaults to 7 days.
## Configuration
You can configure retention policies using CLI flags, environment variables, or
a YAML configuration file.
### Settings
| Setting | CLI Flag | Environment Variable | Default | Description |
|-----------------|-------------------------------|-----------------------------------|----------------|--------------------------------------|
| Audit Logs | `--audit-logs-retention` | `CODER_AUDIT_LOGS_RETENTION` | `0` (disabled) | How long to retain Audit Log entries |
| Connection Logs | `--connection-logs-retention` | `CODER_CONNECTION_LOGS_RETENTION` | `0` (disabled) | How long to retain Connection Logs |
| API Keys | `--api-keys-retention` | `CODER_API_KEYS_RETENTION` | `7d` | How long to retain expired API keys |
### Duration Format
Retention durations support days (`d`) and weeks (`w`) in addition to standard
Go duration units (`h`, `m`, `s`):
- `7d` - 7 days
- `2w` - 2 weeks
- `30d` - 30 days
- `90d` - 90 days
- `365d` - 1 year
### CLI Example
```bash
coder server \
--audit-logs-retention=365d \
--connection-logs-retention=90d \
--api-keys-retention=7d
```
### Environment Variables Example
```bash
export CODER_AUDIT_LOGS_RETENTION=365d
export CODER_CONNECTION_LOGS_RETENTION=90d
export CODER_API_KEYS_RETENTION=7d
```
### YAML Configuration Example
```yaml
retention:
audit_logs: 365d
connection_logs: 90d
api_keys: 7d
```
## How Retention Works
### Background Purge Process
Coder runs a background process that periodically deletes old records. The
purge process:
1. Runs approximately every 10 minutes.
2. Processes records in batches to avoid database lock contention.
3. Deletes records older than the configured retention period.
4. Logs the number of deleted records for monitoring.
### Effective Retention
Each retention setting controls its data type independently:
- When set to a non-zero duration, records older than that duration are deleted.
- When set to `0`, retention is disabled and data is kept indefinitely.
### API Keys Special Behavior
API key retention only affects **expired** keys. A key is deleted only when:
1. The key has expired (past its `expires_at` timestamp).
2. The key has been expired for longer than the retention period.
Setting `--api-keys-retention=7d` deletes keys that expired more than 7 days
ago. Active keys are never deleted by the retention policy.
Keeping expired keys for a short period allows Coder to return a more helpful
error message when users attempt to use an expired key.
## Best Practices
### Recommended Starting Configuration
For most deployments, we recommend:
```yaml
retention:
audit_logs: 365d
connection_logs: 90d
api_keys: 7d
```
### Compliance Considerations
> [!WARNING]
> Audit Logs provide critical security and compliance information. Purging
> Audit Logs may impact your organization's ability to investigate security
> incidents or meet compliance requirements. Consult your security and
> compliance teams before configuring Audit Log retention.
Common compliance frameworks have varying retention requirements:
- **SOC 2**: Typically requires 1 year of audit logs.
- **HIPAA**: Requires 6 years for certain records.
- **PCI DSS**: Requires 1 year of audit logs, with 3 months immediately
available.
- **GDPR**: Requires data minimization but does not specify maximum retention.
### External Log Aggregation
If you use an external log aggregation system (Splunk, Datadog, etc.), you can
configure shorter retention periods in Coder since logs are preserved
externally. See
[Capturing/Exporting Audit Logs](../security/audit-logs.md#capturingexporting-audit-logs)
for details on exporting logs.
### Database Maintenance
After enabling retention policies, you may want to run a `VACUUM` operation on
your PostgreSQL database to reclaim disk space. See
[Maintenance Procedures](../security/audit-logs.md#maintenance-procedures-for-the-audit-logs-table)
for guidance.
## Keeping Data Indefinitely
To keep data indefinitely for any data type, set its retention value to `0`:
```yaml
retention:
audit_logs: 0s # Keep audit logs forever
connection_logs: 0s # Keep connection logs forever
api_keys: 0s # Keep expired API keys forever
```
## Monitoring
The purge process logs deletion counts at the `DEBUG` level. To monitor
retention activity, enable debug logging or search your logs for entries
containing the table name (e.g., `audit_logs`, `connection_logs`, `api_keys`).
## Related Documentation
- [Audit Logs](../security/audit-logs.md): Learn about Audit Logs and manual
purge procedures.
- [Connection Logs](../monitoring/connection-logs.md): Learn about Connection
Logs and monitoring.
docs/manifest.json
+5
View File
@@ -363,6 +363,11 @@
"title": "Telemetry",
"description": "Learn what usage telemetry Coder collects",
"path": "./admin/setup/telemetry.md"
},
{
"title": "Data Retention",
"description": "Configure data retention policies for database tables",
"path": "./admin/setup/data-retention.md"
}
]
},