feat: add premium paywall conversion telemetry (#28425)

**TLDR:** When someone using free Coder taps a "Start trial for free"
button, we now write down which page they tapped it on, and whether they
went on to actually start the trial. That tells us which locked features
make people want Premium.

## Brief summary

- Records two things: the button being clicked, and a trial actually
being started.
- Each click gets a random ID that travels with the person to the trial
form, so we can tell who finished from who walked away.
- Records a short label for the page (like `appearance` or `audit_log`),
never the web address, so no customer names are collected.

#28226 added the trial form and has since merged, so this now targets
`main`.

## Description

Today the "Start trial for free" button just moves you to the Premium
page and nothing is recorded, so there is no way to know which feature
convinced someone.

This adds two events:

| Event | Happens when |
|---|---|
| `cta_click` | Someone clicks "Start trial for free" |
| `trial_signup` | A trial licence is actually issued |

The click generates a random ID. It rides along to the trial form and
gets saved on the signup, so a click and a signup can be matched up
later. Clicks with no matching signup are the people who gave up. Trials
started without a paywall (for example straight from the sidebar) are
labelled `direct`, so "no label" never gets confused with "broken".

A few small choices worth knowing:

- The ID is kept in browser session storage, not in the web address, so
the URL stays clean. It expires after 30 minutes.
- The trial signup event is recorded by the server, not the browser, so
nobody can fake a signup.
- The browser cannot say which kind of event it is sending. It can only
report a click; the server labels it.
- Only people allowed to start a trial are counted. The server rejects
the rest, which matters because a few paywalls show their button to
everyone.
- Nothing is recorded at all when telemetry is turned off.

There is deliberately no "paywall was shown" event. Every admin page
visit would report one, which is a lot of noise for little insight.

Every paywall in the [Premium Copy
Updates](https://www.notion.so/coderhq/Premium-Copy-Updates-3c1d579be59280a59557cabd54fd8a1f)
table has a label. The rows that do not are badges and alerts with no
button to click.

## Proof

Ran a local Coder with no licence and clicked the paywall button on the
Appearance page.

What the browser sent, and the reply:

```json
POST /api/v2/deployment/premium-funnel-events  ->  204 No Content
{"id":"92d96afa-f975-4789-b8a7-b1fd19d4df85","source":"appearance","variant":"premium"}
```

What the browser saved for the trial form, same ID:

```json
{"id":"92d96afa-f975-4789-b8a7-b1fd19d4df85","source":"appearance","createdAt":1787268520857}
```

The click then landed on the Premium page with the trial form.
Screenshots of the network panel, the saved value, and the Premium page
are attached below. The form itself was not submitted, since that would
contact the real licence server.

## Testing

- Backend tests: a click is recorded with the server's own label, an
unknown page label is refused, and a normal member is refused.
- Storybook tests: button clicked and ID saved on all three paywall
styles, plus the case where the viewer only sees guidance text and has
nothing to click.
- Unit tests for the saved ID: expiry, bad data, unknown page label, and
the `direct` fallback.
- `make gen` is clean, and lint, types, and formatting pass.

Two stories fail on `main` today and fail the same way here:
`ExternalAuthSettingsPageView > Page` and `PremiumPageView > No
License`. Neither file is touched by this branch; both were verified
failing on `main` with this branch checked out elsewhere.

## Follow-up

coder/coder-telemetry-server#45 saves these events for reporting. It
merges after this one.

<details>
<summary>Plan and decisions</summary>

**Goal:** find out which Premium paywall drives trial signups, and tell
abandons apart from completions.

**Options considered and dropped:**

1. Put `?from=appearance` in the URL. Rejected: messy for users.
2. Use router state. Rejected: lost if the page is refreshed.
3. Let the browser report the signup. Rejected: it could be faked or
lost during the redirect.
4. Work out the page from the URL. Rejected: URLs contain organization
and template names.
5. Report every paywall impression. Rejected: one event per admin page
visit is noise, and the click is the signal.
6. Separate event types and tables. Rejected: makes the matching query
harder for no gain.

**Event shape:**

```go
type PremiumFunnelEvent struct {
	ID            uuid.UUID // event ID; for a click this is also the matching ID
	EventType     string    // cta_click | trial_signup, set by the server
	Source        string    // appearance, audit_log, ..., direct
	Variant       string    // which paywall design was shown
	AttributionID uuid.UUID // the click this signup came from
	UserID        uuid.UUID
	CreatedAt     time.Time
}
```

**Where the code lives:** nothing under `components/` imports from
`modules/`, so the paywall components stay presentational and only gain
an `onCTAClick` prop. New wrappers in `modules/paywall/` own the
reporting, one per paywall style, each taking a fixed `source`.

**Self-review:** the `frontend-review` skill caught that reading
permissions inside the wrapper forced an auth context into
presentational page views, which broke their stories. Every call site
already passes `canViewPremium={permissions.viewAllLicenses}` and the
button only renders when that is true, so the wrapper no longer reads
permissions and the server stays the gate. It also flagged one
unattributed paywall on the Security page, now wired up as
`browser_only`.

</details>

> Generated with [Coder Agents](https://coder.com/agents) on behalf of
@david-fraley
This commit is contained in:
david-fraley
2026-08-21 10:31:01 -04:00
committed by GitHub
parent f44a3b59d7
commit a2afb458c5
45 changed files with 1339 additions and 52 deletions
+125
View File
@@ -3389,6 +3389,39 @@ const docTemplate = `{
]
}
},
"/api/v2/deployment/premium-funnel-events": {
"post": {
"consumes": [
"application/json"
],
"tags": [
"Enterprise"
],
"summary": "Report a premium paywall click",
"operationId": "report-a-premium-paywall-click",
"parameters": [
{
"description": "Premium funnel event",
"name": "request",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/codersdk.PremiumFunnelEventRequest"
}
}
],
"responses": {
"204": {
"description": "No Content"
}
},
"security": [
{
"CoderSessionToken": []
}
]
}
},
"/api/v2/deployment/ssh": {
"get": {
"produces": [
@@ -19894,6 +19927,11 @@ const docTemplate = `{
"phone_number"
],
"properties": {
"attribution_id": {
"description": "AttributionID is the ID of the cta_click funnel event that led here, so\nthat a signup can be joined back to the paywall that produced it.",
"type": "string",
"format": "uuid"
},
"company_name": {
"type": "string",
"maxLength": 100,
@@ -19936,6 +19974,14 @@ const docTemplate = `{
"maxLength": 20,
"minLength": 7,
"example": "+14155552671"
},
"source": {
"description": "Source is the premium paywall the request came from, for telemetry. It\nis not forwarded to the licensor. Omit it to report \"direct\".",
"allOf": [
{
"$ref": "#/definitions/codersdk.PremiumFunnelSource"
}
]
}
}
},
@@ -23750,6 +23796,85 @@ const docTemplate = `{
}
}
},
"codersdk.PremiumFunnelEventRequest": {
"type": "object",
"required": [
"id",
"source",
"variant"
],
"properties": {
"id": {
"description": "ID identifies this click, and doubles as the attribution token that a\nlater trial signup reports.",
"type": "string",
"format": "uuid"
},
"source": {
"$ref": "#/definitions/codersdk.PremiumFunnelSource"
},
"variant": {
"$ref": "#/definitions/codersdk.PremiumFunnelVariant"
}
}
},
"codersdk.PremiumFunnelSource": {
"type": "string",
"enum": [
"aibridge_session_threads",
"aibridge_sessions",
"ai_gateway_keys",
"ai_governance",
"appearance",
"audit_log",
"browser_only",
"connection_log",
"custom_roles",
"groups",
"idp_org_sync",
"idp_sync",
"multiple_organizations",
"observability",
"provisioner_keys",
"provisioners",
"template_permissions",
"workspace_proxies",
"direct"
],
"x-enum-varnames": [
"PremiumFunnelSourceAIBridgeSessionThreads",
"PremiumFunnelSourceAIBridgeSessions",
"PremiumFunnelSourceAIGatewayKeys",
"PremiumFunnelSourceAIGovernance",
"PremiumFunnelSourceAppearance",
"PremiumFunnelSourceAuditLog",
"PremiumFunnelSourceBrowserOnly",
"PremiumFunnelSourceConnectionLog",
"PremiumFunnelSourceCustomRoles",
"PremiumFunnelSourceGroups",
"PremiumFunnelSourceIdpOrgSync",
"PremiumFunnelSourceIdpSync",
"PremiumFunnelSourceMultipleOrganizations",
"PremiumFunnelSourceObservability",
"PremiumFunnelSourceProvisionerKeys",
"PremiumFunnelSourceProvisioners",
"PremiumFunnelSourceTemplatePermissions",
"PremiumFunnelSourceWorkspaceProxies",
"PremiumFunnelSourceDirect"
]
},
"codersdk.PremiumFunnelVariant": {
"type": "string",
"enum": [
"premium",
"small",
"ai_governance"
],
"x-enum-varnames": [
"PremiumFunnelVariantPremium",
"PremiumFunnelVariantSmall",
"PremiumFunnelVariantAIGovernance"
]
},
"codersdk.Preset": {
"type": "object",
"properties": {
+113
View File
@@ -3008,6 +3008,35 @@
]
}
},
"/api/v2/deployment/premium-funnel-events": {
"post": {
"consumes": ["application/json"],
"tags": ["Enterprise"],
"summary": "Report a premium paywall click",
"operationId": "report-a-premium-paywall-click",
"parameters": [
{
"description": "Premium funnel event",
"name": "request",
"in": "body",
"required": true,
"schema": {
"$ref": "#/definitions/codersdk.PremiumFunnelEventRequest"
}
}
],
"responses": {
"204": {
"description": "No Content"
}
},
"security": [
{
"CoderSessionToken": []
}
]
}
},
"/api/v2/deployment/ssh": {
"get": {
"produces": ["application/json"],
@@ -18011,6 +18040,11 @@
"phone_number"
],
"properties": {
"attribution_id": {
"description": "AttributionID is the ID of the cta_click funnel event that led here, so\nthat a signup can be joined back to the paywall that produced it.",
"type": "string",
"format": "uuid"
},
"company_name": {
"type": "string",
"maxLength": 100,
@@ -18053,6 +18087,14 @@
"maxLength": 20,
"minLength": 7,
"example": "+14155552671"
},
"source": {
"description": "Source is the premium paywall the request came from, for telemetry. It\nis not forwarded to the licensor. Omit it to report \"direct\".",
"allOf": [
{
"$ref": "#/definitions/codersdk.PremiumFunnelSource"
}
]
}
}
},
@@ -21743,6 +21785,77 @@
}
}
},
"codersdk.PremiumFunnelEventRequest": {
"type": "object",
"required": ["id", "source", "variant"],
"properties": {
"id": {
"description": "ID identifies this click, and doubles as the attribution token that a\nlater trial signup reports.",
"type": "string",
"format": "uuid"
},
"source": {
"$ref": "#/definitions/codersdk.PremiumFunnelSource"
},
"variant": {
"$ref": "#/definitions/codersdk.PremiumFunnelVariant"
}
}
},
"codersdk.PremiumFunnelSource": {
"type": "string",
"enum": [
"aibridge_session_threads",
"aibridge_sessions",
"ai_gateway_keys",
"ai_governance",
"appearance",
"audit_log",
"browser_only",
"connection_log",
"custom_roles",
"groups",
"idp_org_sync",
"idp_sync",
"multiple_organizations",
"observability",
"provisioner_keys",
"provisioners",
"template_permissions",
"workspace_proxies",
"direct"
],
"x-enum-varnames": [
"PremiumFunnelSourceAIBridgeSessionThreads",
"PremiumFunnelSourceAIBridgeSessions",
"PremiumFunnelSourceAIGatewayKeys",
"PremiumFunnelSourceAIGovernance",
"PremiumFunnelSourceAppearance",
"PremiumFunnelSourceAuditLog",
"PremiumFunnelSourceBrowserOnly",
"PremiumFunnelSourceConnectionLog",
"PremiumFunnelSourceCustomRoles",
"PremiumFunnelSourceGroups",
"PremiumFunnelSourceIdpOrgSync",
"PremiumFunnelSourceIdpSync",
"PremiumFunnelSourceMultipleOrganizations",
"PremiumFunnelSourceObservability",
"PremiumFunnelSourceProvisionerKeys",
"PremiumFunnelSourceProvisioners",
"PremiumFunnelSourceTemplatePermissions",
"PremiumFunnelSourceWorkspaceProxies",
"PremiumFunnelSourceDirect"
]
},
"codersdk.PremiumFunnelVariant": {
"type": "string",
"enum": ["premium", "small", "ai_governance"],
"x-enum-varnames": [
"PremiumFunnelVariantPremium",
"PremiumFunnelVariantSmall",
"PremiumFunnelVariantAIGovernance"
]
},
"codersdk.Preset": {
"type": "object",
"properties": {
+1
View File
@@ -1578,6 +1578,7 @@ func New(options *Options) *API {
r.Get("/config", api.deploymentValues)
r.Get("/stats", api.deploymentStats)
r.Get("/ssh", api.sshConfig)
r.Post("/premium-funnel-events", api.postPremiumFunnelEvent)
})
r.Route("/experiments", func(r chi.Router) {
r.Use(apiKeyMiddleware)
+75
View File
@@ -0,0 +1,75 @@
package coderd
import (
"net/http"
"github.com/coder/coder/v2/coderd/database/dbtime"
"github.com/coder/coder/v2/coderd/httpapi"
"github.com/coder/coder/v2/coderd/httpmw"
"github.com/coder/coder/v2/coderd/rbac"
"github.com/coder/coder/v2/coderd/rbac/policy"
"github.com/coder/coder/v2/coderd/telemetry"
"github.com/coder/coder/v2/codersdk"
)
// @Summary Report a premium paywall click
// @ID report-a-premium-paywall-click
// @Security CoderSessionToken
// @Accept json
// @Tags Enterprise
// @Param request body codersdk.PremiumFunnelEventRequest true "Premium funnel event"
// @Success 204
// @Router /api/v2/deployment/premium-funnel-events [post]
func (api *API) postPremiumFunnelEvent(rw http.ResponseWriter, r *http.Request) {
ctx := r.Context()
apiKey := httpmw.APIKey(r)
// Reports are authorized rather than trusted: a few paywalls show their
// call to action to everyone, so a user who cannot read licenses can click
// one, and their click must not enter the funnel.
if !api.Authorize(r, policy.ActionRead, rbac.ResourceLicense) {
httpapi.ResourceNotFound(rw)
return
}
var req codersdk.PremiumFunnelEventRequest
if !httpapi.Read(ctx, rw, r, &req) {
return
}
var validations []codersdk.ValidationError
if !req.Source.Valid() {
validations = append(validations, codersdk.ValidationError{
Field: "source",
Detail: "Unknown premium funnel source.",
})
}
if !req.Variant.Valid() {
validations = append(validations, codersdk.ValidationError{
Field: "variant",
Detail: "Unknown premium paywall variant.",
})
}
if len(validations) > 0 {
httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{
Message: "Invalid premium funnel event.",
Validations: validations,
})
return
}
api.Telemetry.Report(&telemetry.Snapshot{
PremiumFunnelEvents: []telemetry.PremiumFunnelEvent{
{
ID: req.ID,
EventType: telemetry.PremiumFunnelEventCTAClick,
Source: string(req.Source),
Variant: string(req.Variant),
UserID: apiKey.UserID,
CreatedAt: dbtime.Now(),
},
},
})
rw.WriteHeader(http.StatusNoContent)
}
+103
View File
@@ -0,0 +1,103 @@
package coderd_test
import (
"context"
"net/http"
"testing"
"github.com/google/uuid"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/coderd/telemetry"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/testutil"
)
func TestPremiumFunnelEvent(t *testing.T) {
t.Parallel()
t.Run("CTAClick", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitMedium)
fTelemetry := newFakeTelemetryReporter(ctx, t, 10)
client := coderdtest.New(t, &coderdtest.Options{
TelemetryReporter: fTelemetry,
})
_ = coderdtest.CreateFirstUser(t, client)
eventID := uuid.New()
err := client.ReportPremiumFunnelEvent(ctx, codersdk.PremiumFunnelEventRequest{
ID: eventID,
Source: codersdk.PremiumFunnelSourceAuditLog,
Variant: codersdk.PremiumFunnelVariantAIGovernance,
})
require.NoError(t, err)
event := receivePremiumFunnelEvent(ctx, t, fTelemetry)
require.Equal(t, eventID, event.ID)
require.Equal(t, telemetry.PremiumFunnelEventCTAClick, event.EventType)
require.Equal(t, string(codersdk.PremiumFunnelSourceAuditLog), event.Source)
require.Equal(t, string(codersdk.PremiumFunnelVariantAIGovernance), event.Variant)
// Only trial signups carry an attribution.
require.Equal(t, uuid.Nil, event.AttributionID)
require.NotEqual(t, uuid.Nil, event.UserID)
require.False(t, event.CreatedAt.IsZero())
})
t.Run("UnknownSource", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitMedium)
client := coderdtest.New(t, nil)
_ = coderdtest.CreateFirstUser(t, client)
err := client.ReportPremiumFunnelEvent(ctx, codersdk.PremiumFunnelEventRequest{
ID: uuid.New(),
Source: "/organizations/acme-corp/idp-sync",
Variant: codersdk.PremiumFunnelVariantPremium,
})
require.Error(t, err)
var sdkErr *codersdk.Error
require.ErrorAs(t, err, &sdkErr)
require.Equal(t, http.StatusBadRequest, sdkErr.StatusCode())
require.Len(t, sdkErr.Validations, 1)
require.Equal(t, "source", sdkErr.Validations[0].Field)
})
t.Run("MemberCannotReport", func(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitMedium)
client := coderdtest.New(t, nil)
admin := coderdtest.CreateFirstUser(t, client)
memberClient, _ := coderdtest.CreateAnotherUser(t, client, admin.OrganizationID)
err := memberClient.ReportPremiumFunnelEvent(ctx, codersdk.PremiumFunnelEventRequest{
ID: uuid.New(),
Source: codersdk.PremiumFunnelSourceAppearance,
Variant: codersdk.PremiumFunnelVariantPremium,
})
require.Error(t, err)
var sdkErr *codersdk.Error
require.ErrorAs(t, err, &sdkErr)
require.Equal(t, http.StatusNotFound, sdkErr.StatusCode())
})
}
// receivePremiumFunnelEvent drains snapshots until one carries a funnel event.
// Creating the first user reports its own snapshots on the same channel.
func receivePremiumFunnelEvent(ctx context.Context, t *testing.T, reporter *fakeTelemetryReporter) telemetry.PremiumFunnelEvent {
t.Helper()
for {
snapshot := testutil.TryReceive(ctx, t, reporter.snapshots)
if len(snapshot.PremiumFunnelEvents) > 0 {
require.Len(t, snapshot.PremiumFunnelEvents, 1)
return snapshot.PremiumFunnelEvents[0]
}
}
}
+23
View File
@@ -1640,6 +1640,7 @@ type Snapshot struct {
ChatDiffStatusSummary *ChatDiffStatusSummary `json:"chat_diff_status_summary"`
UserSecretsSummary *UserSecretsSummary `json:"user_secrets_summary"`
TemplateBuilderSessions []TemplateBuilderSession `json:"template_builder_sessions"`
PremiumFunnelEvents []PremiumFunnelEvent `json:"premium_funnel_events"`
}
// Deployment contains information about the host running Coder.
@@ -2677,6 +2678,28 @@ type TemplateBuilderSession struct {
CreatedAt time.Time `json:"created_at"`
}
// Steps of the premium trial funnel. These are set by coderd rather than the
// client so that a conversion cannot be forged.
const (
PremiumFunnelEventCTAClick = "cta_click"
PremiumFunnelEventTrialSignup = "trial_signup"
)
// PremiumFunnelEvent tracks a single step of the premium trial funnel: a
// paywall call to action being clicked, or a trial license being issued.
// AttributionID carries the ID of the cta_click event that led to a trial
// signup, so signups can be joined back to the paywall that produced them; it
// is the nil UUID for clicks and for trials started without a paywall.
type PremiumFunnelEvent struct {
ID uuid.UUID `json:"id"`
EventType string `json:"event_type"`
Source string `json:"source"`
Variant string `json:"variant"`
AttributionID uuid.UUID `json:"attribution_id"`
UserID uuid.UUID `json:"user_id"`
CreatedAt time.Time `json:"created_at"`
}
func ConvertAIBridgeInterceptionsSummary(endTime time.Time, provider, model, client string, summary database.CalculateAIBridgeInterceptionsTelemetrySummaryRow) AIBridgeInterceptionsSummary {
return AIBridgeInterceptionsSummary{
ID: uuid.New(),