Files
sim/scripts
Vikhyath Mondreti a25d993266 improvement(billing): ledger-only usage + period-advance cycle close (#7078)
* improvement(billing): make usage ledger-only and close cycles off period advance

Usage is now the attributed usage_log ledger everywhere: the userStats
baselines (currentPeriodCost / currentPeriodCopilotCost), the
includeLegacyBaseline compatibility flag, the pro-snapshot join/leave
machinery, and departedMemberUsage accrual are removed from all read and
write paths. Cycle rollover (final sub-threshold overage collection,
billedOverageThisPeriod reset, last-period bookkeeping) moves off the
invoice.finalized payload parsing — dead for org subscriptions since May —
onto a period-advance sweep with a durable per-subscription close marker
(subscription.last_closed_period_start), transaction-enlisted Stripe
outbox invoicing, and stamp-matched ledger sums. Enterprise closes are
bookkeeping-only; reporting-anchor orgs advance the marker alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): include departed actors in close refresh and gate threshold billing on close currency

Cycle close now unions current members with every actor holding org-stamped
ledger rows in the closed period, so a departed member's daily-refresh
consumption offsets the final overage exactly like their billed usage.
Threshold billing defers with a pending-cycle-close no-op while a
subscription's close marker lags its current period, so the shared
billedOverageThisPeriod tracker can never mix an elapsed period's
settlements with the new period's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): seal cycle-close races and align refresh actors with billed usage

Threshold settlement revalidates the close marker and period under the
tracker lock via the same isSubscriptionCycleCloseCurrent predicate the
preflight uses, so a rollover between check and transaction aborts as a
concurrent-state retry instead of settling against the wrong cycle.
Terminal bookkeeping claims the close marker with its tracker reset, so a
deletion racing an in-flight sweep close serializes through the one marker
and the loser rolls back rather than re-billing settled overage; anchored
enterprise deletions only claim the marker since their windows derive from
the anchor, not Stripe bounds. A close with overage due but missing Stripe
identifiers now defers loudly instead of claiming the marker and silently
forgiving the money, the closed window's start derives from the ledger's
own period stamps so anchor-day drift cannot misalign the refresh window,
calculateSubscriptionOverage unions departed ledger actors into the org
refresh deduction like the close does, and blocked accounts report their
real ledger usage while staying blocked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): defer ownerless org closes and make the drift test load-bearing

A close with overage due but no owner-role member now defers loudly like
the missing-Stripe-identifier case instead of claiming the marker and
silently forgiving the money. The stamp-drift test pins the marker before
the stamped boundary so only the ledger-stamp lookup can produce the
asserted window.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(helm): bump chart to 1.6.0 for the billing-cycle-close cron job

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): revalidate the org roster under close locks and align invoice labels

The close re-reads the member roster inside its transaction and defers on
any change, mirroring threshold billing — an owner transfer moves the
billed-overage tracker between rows, so a pre-lock roster could settle
against the wrong tracker. Invoice labels now use the closed period's end
month like every other overage path, and the sweep test's rows are shaped
like rows the candidate query can actually return.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): hold cycle close for a settlement grace after rollover

Billing attribution is frozen at run start, so a run straddling a
rollover can insert elapsed-period-stamped rows after the period ends.
Closing only once the rollover is older than any possible in-flight run
guarantees the close's ledger sums are final; the sweep picks the period
up on a later run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): resolve reporting windows through the canonical period resolver

The close paths now ask resolveSubscriptionUsagePeriod whether a
subscription derives its windows from a reporting anchor instead of
re-checking metadata shape locally, so a malformed hand-edited anchor that
the resolver rejects (falling back to Stripe bounds) books its
Stripe-stamped ledger rows normally instead of skipping bookkeeping.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): union departed ledger actors in org threshold settlement

Organization threshold billing now reads the period ledger per user and
unions the actors holding org-attributed rows with the current roster
before computing refresh deductions — the same actor set
calculateSubscriptionOverage and the cycle close use — so a departed
member's usage cannot be settled without their daily-refresh offset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): claim the terminal period before deletion settlement and delete vestigial refresh bounds

Subscription deletion now claims the close marker from the fresh
subscription row before computing or charging final overage, serializing
with the cycle-close sweep so both paths can never bill the same period —
an in-flight close fails its guarded claim and rolls back, and the
deletion settles against the row's real period instead of a possibly
stale webhook payload. The per-user refresh bounds machinery is deleted
outright: its only source was proPeriodCostSnapshotAt, which this PR
stopped writing, and ledger entity stamps already scope refresh to
org-attributed rows — a joiner's pre-join usage is user-stamped and can
never enter the org refresh scan, while a departed member's org-stamped
rows participate exactly like a current member's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(billing): scope daily refresh by entity stamps and close lagging periods before deletion settlement

The refresh actor-list threading (userIds + departed-actor unions) violated
the entity-stamp principle the ledger sums already follow: org-attributed
rows from departed members counted in pooled usage but never consumed
refresh on the monitor/resolved-usage paths. Daily refresh now scopes rows
by the same write-time entity and period stamps as the ledger — no actor
list anywhere — which deletes the unions, the rollup memberIds, and the
org-specific refresh variant.

Deletion settlement now closes any elapsed-but-unclosed period (grace
bypassed — no later sweep revisits a canceled sub) before claiming the
terminal period, so a deletion racing the sweep can no longer jump the
marker past an unsettled period and silently forgive its final overage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(billing): bucket refresh by clamped day so stamped stragglers stay in the deduction

Refresh membership is now the entity/period stamps alone — identical to the
ledger sums it offsets. A row written after the rollover but stamped to the
elapsed period (attribution frozen at run start) is billed by the stamp-based
close, so it must consume refresh too; created-at now only assigns the day
bucket, clamped into the period, instead of excluding the row entirely.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(billing): pair the overage tracker with the marker's period and pin test clocks

billedOverageThisPeriod only ever holds collections for the period that
began at the close marker — the threshold gate blocks settlement whenever
the marker lags. Both consumers now honor that pairing: a close that
skipped forgiven periods counts nothing from the tracker against the
period it bills, and the deletion settlement ignores the tracker when the
marker was still lagging at claim time. Ignoring is provably safe in both
cases because a lagging marker means no current-period collections exist.

The cycle-close and daily-refresh suites pin the system clock: their grace
and window checks compare fixed period fixtures against Date.now(), which
made them dependent on the host date.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(billing): reject lagging terminal claims and thread billingInterval into deletion bookkeeping

claimTerminalPeriod no longer advances the marker over an unclosed elapsed
period: a lagging marker is reported without a write so the deletion handler
can run the elapsed close once more (healing a rollover that committed
between close and claim) and only then seal the marker explicitly, with an
error log, when the period is genuinely unclosable. Sealing preserves the
in-flight-sweep abort guarantee.

Deletion bookkeeping now passes the subscription's billingInterval through,
so an enterprise reporting subscription whose interval lives on the row
column (not metadata) still resolves as reporting-anchored and keeps its
bookkeeping no-op, matching every other resolver call site.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:34:44 -07:00
..

Integration documentation generator

generate-docs.ts compiles the per-service integration pages under apps/docs/content/docs/en/integrations/ from the block/tool/trigger registry in apps/sim. The ontology it encodes: everything is a block, and an integration is one block that has Actions and, optionally, a Trigger.

Golden rule: the generated .mdx files are derived artifacts, not the source of truth. Do not hand-edit them — your changes are overwritten on the next run. The only editable region is the MANUAL-CONTENT block (see below). To change what a page says, edit the TypeScript in apps/sim and regenerate.

Where an integration lives canonically

For a service like Gmail, three TS sources define it:

Source What it is What it feeds in the page
apps/sim/blocks/blocks/<service>.ts The block: type, name, category (tools for integrations), bgColor, config sub-blocks, tools.access (which actions it exposes), an optional triggers capability, outputs Header / BlockInfoCard, Usage Instructions, and which actions + trigger appear
apps/sim/tools/<service>/*.ts Each action's params + outputs Every ### <action>#### Input / #### Output under ## Actions
apps/sim/triggers/<provider>/ The trigger's config fields + outputs The ## Triggers section
apps/sim/components/icons.tsx The brand glyph The page icon

The block references actions by id in tools.access; the generator looks each one up in apps/sim/tools/.

What the generator does

Run with cd apps/sim && bun run generate-docs (or bun run scripts/generate-docs.ts from the repo root). One pass (generateAllBlockDocs):

  1. Copies icons apps/sim/components/icons.tsxapps/docs/components/icons.tsx and builds apps/docs/components/ui/icon-mapping.ts.
  2. Block pass — for each integration block (category: 'tools', plus the memory / knowledge / table exceptions), writes integrations/<service>.mdx: BlockInfoCard + Usage Instructions + ## Actions.
  3. Trigger pass (generateAllTriggerDocs) — reads apps/sim/triggers/<provider>/ and appends a ## Triggers section to that service's page, or writes a standalone page for trigger-only services.
  4. Writes integrations/meta.json and regenerates the landing page's integrations.json.

Hand-written pages it never touches

Core block pages (blocks/*), the native trigger pages (triggers/{start,schedule,webhook,rss,table}), the integrations overview (integrations/index.mdx), and the service-account pages are fully hand-written. The generator skips them via HANDWRITTEN_INTEGRATION_DOCS, HANDWRITTEN_TRIGGER_DOCS, and SKIP_TRIGGER_PROVIDERS. Add a page name to those sets if you hand-author a page the generator would otherwise produce.

Manual content (the one editable region)

Each generated page may carry hand-written prose inside marker comments. The generator preserves anything between the markers and overwrites everything else, so this survives every regeneration:

{/* MANUAL-CONTENT-START:intro */}
[AgentMail](https://agentmail.to/) is an API-first email platform…
{/* MANUAL-CONTENT-END */}

Supported section names: intro (after the BlockInfoCard — the most common), usage, configuration, outputs, notes. The merge is by marker name (extractManualContent + mergeWithManualContent), so a section is re-inserted at the matching spot in the freshly generated structure.

If you move the output folder, reseed manual content from the old location first — the generator only preserves markers it finds in the existing output file, so a fresh folder starts with none.

Practical: to change…

  • An action's params/outputs, a trigger, or to add a service → edit apps/sim/{blocks,tools,triggers} and re-run the generator.
  • A page's prose intro → edit its MANUAL-CONTENT:intro block directly; it survives regen.
  • The overview / service-account / core-block / native-trigger pages → hand-edit freely.

Gotchas

  • Never hand-edit apps/docs/components/icons.tsx — step 1 overwrites it from the sim app. Components that need an icon the sim app lacks should define it locally or use @sim/emcn/icons (see components/workflow-preview/block-icons.tsx).
  • The generator is the source of truth for integrations/ and its meta.json; manual edits there are transient.

CI

The generator runs in CI on pushes to the main branch and commits the regenerated docs back. Keep block/tool/trigger metadata accurate in apps/sim and the docs follow.