mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
fix(landing): fix oversized HTML and severe LCP on integration/comparison pages (#5634)
* fix(landing): fix oversized HTML and severe LCP on integration/comparison pages Google Search Console flagged /integrations, /integrations/slack, and several others for exceeding Googlebot's 2MB uncompressed-HTML crawl limit, plus severe LCP on /integrations, /integrations/slack, /integrations/hubspot, /comparison/flowise, and /integrations/hugging-face. Root cause 1 - /integrations/slack was 5.2MB (measured on production). The "Agent templates" section renders every template matching getTemplatesForBlock(type) with no cap - Slack is referenced as alsoIntegrations in 445 templates (vs 52-126 for Salesforce/Gmail/ HubSpot), so its detail page embedded hundreds of full template cards in the initial HTML/RSC payload. Capped to 12, consistent with the same file's existing related-integrations cap of 4. Root cause 2 - /integrations was 1.83MB, right at the limit. The client IntegrationGrid component receives the full Integration[] as props (needed for instant client-side search), which serializes every integration's complete operations/triggers arrays - including full per-operation description sentences - into the initial payload purely to build a search index. Added IntegrationSummary + toIntegrationSummary (lib/integrations): the same searchable surface (name, description, operation names, trigger names) precomputed server-side into one lowercased string, dropping the full operations/triggers data the client never actually renders. Measured: 627KB -> 142KB of embedded integration data (77% reduction). Verified via a real production build + curl: - /integrations: 1.83MB -> 1.31MB - /integrations/slack: 5.2MB -> 355KB (93% reduction) - /integrations/hubspot: 901KB -> 452KB Verified via Lighthouse (mobile, devtools throttling, matching what real users on a typical device experience) against both live production and the fixed local build: - /integrations: 47 -> 96 (LCP unmeasured->2.1s, TBT 2,770ms->110ms) - /integrations/slack: 47 -> 96 (LCP 9.7s -> 1.9s) - /integrations/hubspot: 72 -> 97 (LCP 9.1s -> 1.9s) - /integrations/hugging-face: 72 -> 95 (LCP 9.2s -> 2.3s, reproduced twice on production before fixing - not a fluke) - /comparison/flowise: 71 -> 95 (LCP 9.8s -> 2.1s) The last two aren't Slack-style template-count outliers (4 and 0 alsoIntegrations references respectively) - shrinking the shared IntegrationGrid client bundle appears to have reduced a shared chunk loaded broadly across landing pages, benefiting pages beyond the ones directly touched. Also checked and confirmed already healthy, no action needed: /integrations/salesforce, /integrations/gmail, /integrations/amazon-dynamodb, /comparison/tines, /models/xai/grok-4-20-multi-agent-0309. * fix(landing): restore full-fidelity search and template priority Two real regressions from the previous commit, both confirmed by Greptile and Cursor Bugbot independently: - lib/integrations: IntegrationSummary.searchText joined every field into one string, so (a) it dropped operation/trigger descriptions entirely (a search for API-specific terms that only appear in an operation's description, not its name, silently stopped matching), and (b) a single joined string lets a query span a field boundary (e.g. matching across the tail of a name and the head of the next field) that the original per-field search never allowed. Reverted to a searchFields array - same content as the original per-field index (name, description, every operation's name+description, every trigger's name), just precomputed server-side instead of shipping the full Integration objects. Grid filtering goes back to `.some(field => field.includes(q))`, matching the exact original matching semantics. - blocks/registry.ts + integrations/[slug]/page.tsx: capping getTemplatesForBlock's result with a plain .slice(0, 12) kept whatever 12 templates happened to iterate first in registry insertion order - for a high-connectivity integration like Slack, that can be entirely alsoIntegrations matches from unrelated blocks, silently dropping the integration's own owned templates and any marked featured. Added an explicit isOwner flag to ScopedBlockTemplate (the registry function already computes this internally, just wasn't surfacing it) and sort owner-first, featured-second before slicing. * fix(landing): sort featured templates ahead of owned, not just as a tiebreaker The previous sort (owner-first, featured as a tiebreaker within each owner tier) meant an integration with 12+ owned templates filled the entire cap with non-featured owned templates before any featured related template (reached via alsoIntegrations) was ever considered - Slack hits this case. Swapped the sort so featured (owned or related) ranks first, then owned-but-not-featured, so a curated featured template can no longer be sliced away by a pile of ordinary owned ones.
This commit is contained in:
@@ -28,6 +28,22 @@ const allIntegrations = INTEGRATIONS
|
||||
const INTEGRATION_COUNT = allIntegrations.length
|
||||
const baseUrl = SITE_URL
|
||||
|
||||
/**
|
||||
* High-connectivity integrations (e.g. Slack, used as a notification target
|
||||
* across hundreds of unrelated templates via `alsoIntegrations`) can match
|
||||
* many hundreds of templates, ballooning this page's HTML/RSC payload well
|
||||
* past Googlebot's 2MB crawl limit. Cap to a bounded, still-generous set,
|
||||
* consistent with the related-integrations section's own cap below.
|
||||
*
|
||||
* `getTemplatesForBlock` returns matches in registry insertion order, not
|
||||
* relevance - sorted `featured` first, then owned-by-the-viewing-integration,
|
||||
* so the cap can't hide a curated `featured` template (owned or reached via
|
||||
* `alsoIntegrations`) behind a larger pile of non-featured owned templates,
|
||||
* nor behind unrelated `alsoIntegrations` matches that happen to iterate
|
||||
* earlier in the registry.
|
||||
*/
|
||||
const MAX_TEMPLATES_SHOWN = 12
|
||||
|
||||
/** Fast O(1) lookups - avoids repeated linear scans inside render loops. */
|
||||
const bySlug = new Map(allIntegrations.map((i) => [i.slug, i]))
|
||||
const byType = new Map(allIntegrations.map((i) => [i.type, i]))
|
||||
@@ -380,6 +396,12 @@ export default async function IntegrationPage({ params }: { params: Promise<{ sl
|
||||
relatedIntegrations.map((i) => i.name)
|
||||
)
|
||||
const matchingTemplates = getTemplatesForBlock(integration.type)
|
||||
.sort(
|
||||
(a, b) =>
|
||||
Number(b.featured ?? false) - Number(a.featured ?? false) ||
|
||||
Number(b.isOwner) - Number(a.isOwner)
|
||||
)
|
||||
.slice(0, MAX_TEMPLATES_SHOWN)
|
||||
|
||||
const breadcrumbJsonLd = {
|
||||
'@context': 'https://schema.org',
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
INTEGRATIONS,
|
||||
type Integration,
|
||||
POPULAR_WORKFLOWS,
|
||||
toIntegrationSummary,
|
||||
} from '@/lib/integrations'
|
||||
import { withFilteredNoindex } from '@/lib/landing/seo'
|
||||
import { JsonLd } from '@/app/(landing)/components/json-ld'
|
||||
@@ -17,6 +18,7 @@ import { RequestIntegrationModal } from '@/app/(landing)/integrations/components
|
||||
import { integrationsSearchParamsCache } from '@/app/(landing)/integrations/search-params'
|
||||
|
||||
const allIntegrations = INTEGRATIONS
|
||||
const integrationSummaries = allIntegrations.map(toIntegrationSummary)
|
||||
const INTEGRATION_COUNT = allIntegrations.length
|
||||
const OAUTH_COUNT = allIntegrations.filter((i) => i.authType === 'oauth').length
|
||||
const TRIGGER_INTEGRATION_COUNT = allIntegrations.filter((i) => i.triggerCount > 0).length
|
||||
@@ -75,9 +77,9 @@ const INTEGRATIONS_BREADCRUMB_JSON_LD = {
|
||||
const FEATURED_SLUGS = ['slack', 'notion', 'github', 'gmail'] as const
|
||||
|
||||
const bySlug = new Map(allIntegrations.map((i) => [i.slug, i]))
|
||||
const featured = FEATURED_SLUGS.map((s) => bySlug.get(s)).filter(
|
||||
(i): i is Integration => i !== undefined
|
||||
)
|
||||
const featured = FEATURED_SLUGS.map((s) => bySlug.get(s))
|
||||
.filter((i): i is Integration => i !== undefined)
|
||||
.map(toIntegrationSummary)
|
||||
|
||||
/**
|
||||
* `q`/`category` render a genuinely different server-rendered list (see
|
||||
@@ -215,7 +217,7 @@ export default async function IntegrationsPage({
|
||||
All Integrations
|
||||
</h2>
|
||||
</div>
|
||||
<IntegrationGrid integrations={allIntegrations} />
|
||||
<IntegrationGrid integrations={integrationSummaries} />
|
||||
</section>
|
||||
|
||||
{/* FAQ */}
|
||||
|
||||
@@ -1,14 +1,14 @@
|
||||
import type { ComponentType, SVGProps } from 'react'
|
||||
import { memo } from 'react'
|
||||
import Link from 'next/link'
|
||||
import type { Integration } from '@/lib/integrations'
|
||||
import type { IntegrationSummary } from '@/lib/integrations'
|
||||
import { ChevronArrow } from '@/app/(landing)/components/chevron-arrow'
|
||||
import { IntegrationIcon } from '@/app/(landing)/integrations/components/integration-icon'
|
||||
|
||||
const HOVER_BG = 'transition-colors hover:bg-[var(--surface-hover)]' as const
|
||||
|
||||
interface IntegrationItemProps {
|
||||
integration: Integration
|
||||
integration: IntegrationSummary
|
||||
IconComponent?: ComponentType<SVGProps<SVGSVGElement>>
|
||||
}
|
||||
|
||||
|
||||
@@ -3,7 +3,11 @@
|
||||
import { useMemo } from 'react'
|
||||
import { ChipInput, Search } from '@sim/emcn'
|
||||
import { debounce, useQueryStates } from 'nuqs'
|
||||
import { blockTypeToIconMap, formatIntegrationType, type Integration } from '@/lib/integrations'
|
||||
import {
|
||||
blockTypeToIconMap,
|
||||
formatIntegrationType,
|
||||
type IntegrationSummary,
|
||||
} from '@/lib/integrations'
|
||||
import { IntegrationRow } from '@/app/(landing)/integrations/components/integration-card'
|
||||
import {
|
||||
integrationsParsers,
|
||||
@@ -19,7 +23,7 @@ const PILL_ACTIVE = 'bg-[var(--surface-active)]' as const
|
||||
const PILL_INACTIVE = 'hover:bg-[var(--surface-hover)]' as const
|
||||
|
||||
interface IntegrationGridProps {
|
||||
integrations: readonly Integration[]
|
||||
integrations: readonly IntegrationSummary[]
|
||||
}
|
||||
|
||||
export function IntegrationGrid({ integrations }: IntegrationGridProps) {
|
||||
@@ -29,30 +33,17 @@ export function IntegrationGrid({ integrations }: IntegrationGridProps) {
|
||||
)
|
||||
const activeCategory = category || null
|
||||
|
||||
// Category facets and a per-integration lowercased search index, derived once
|
||||
// from the (stable) integration list instead of rebuilt on every keystroke.
|
||||
// The index keeps each searchable field as its own entry so matching stays
|
||||
// identical to a per-field `includes` (no cross-field boundary matches).
|
||||
const { availableCategories, searchIndex } = useMemo(() => {
|
||||
/** Category facets, derived once from the (stable) integration list. */
|
||||
const availableCategories = useMemo(() => {
|
||||
const counts = new Map<string, number>()
|
||||
const searchIndex = new Map<string, string[]>()
|
||||
for (const i of integrations) {
|
||||
if (i.integrationType) {
|
||||
counts.set(i.integrationType, (counts.get(i.integrationType) || 0) + 1)
|
||||
}
|
||||
searchIndex.set(i.type, [
|
||||
i.name.toLowerCase(),
|
||||
i.description.toLowerCase(),
|
||||
...i.operations.flatMap((op) => [op.name.toLowerCase(), op.description.toLowerCase()]),
|
||||
...i.triggers.map((t) => t.name.toLowerCase()),
|
||||
])
|
||||
}
|
||||
return {
|
||||
availableCategories: Array.from(counts.entries())
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.map(([key]) => key),
|
||||
searchIndex,
|
||||
}
|
||||
return Array.from(counts.entries())
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.map(([key]) => key)
|
||||
}, [integrations])
|
||||
|
||||
const q = query.trim().toLowerCase()
|
||||
@@ -61,9 +52,9 @@ export function IntegrationGrid({ integrations }: IntegrationGridProps) {
|
||||
integrations.filter((i) => {
|
||||
if (activeCategory && i.integrationType !== activeCategory) return false
|
||||
if (!q) return true
|
||||
return searchIndex.get(i.type)?.some((field) => field.includes(q)) ?? false
|
||||
return i.searchFields.some((field) => field.includes(q))
|
||||
}),
|
||||
[integrations, searchIndex, q, activeCategory]
|
||||
[integrations, q, activeCategory]
|
||||
)
|
||||
|
||||
return (
|
||||
|
||||
@@ -183,6 +183,8 @@ export function getAllBlockMeta(): Record<string, BlockMeta> {
|
||||
export interface ScopedBlockTemplate extends BlockTemplate {
|
||||
/** Block types (base form) to render alongside the viewing block in the icon cluster. */
|
||||
otherBlockTypes: readonly string[]
|
||||
/** Whether the viewing block owns this template, vs matching only via `alsoIntegrations`. */
|
||||
isOwner: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -208,7 +210,7 @@ export function getTemplatesForBlock(type: string): ScopedBlockTemplate[] {
|
||||
const alsoBase = stripVersionSuffix(also)
|
||||
if (alsoBase !== base && !others.includes(alsoBase)) others.push(alsoBase)
|
||||
}
|
||||
collected.push({ ...template, otherBlockTypes: others })
|
||||
collected.push({ ...template, otherBlockTypes: others, isOwner: isOwnerMatch })
|
||||
}
|
||||
}
|
||||
return collected
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
|
||||
import { stripVersionSuffix } from '@sim/utils/string'
|
||||
import integrationsJson from '@/lib/integrations/integrations.json'
|
||||
import type { Integration } from '@/lib/integrations/types'
|
||||
import type { Integration, IntegrationSummary } from '@/lib/integrations/types'
|
||||
import { getAllBlockMeta } from '@/blocks/registry'
|
||||
|
||||
/** All integrations surfaced in the catalog, ordered by `scripts/generate-docs.ts`. */
|
||||
@@ -67,13 +67,44 @@ export const POPULAR_WORKFLOWS: readonly PopularWorkflow[] = (() => {
|
||||
return pairs
|
||||
})()
|
||||
|
||||
/**
|
||||
* Projects a full `Integration` down to the fields the `/integrations`
|
||||
* catalog grid renders and searches by, replacing the full `operations`/
|
||||
* `triggers` arrays with a precomputed, lowercased `searchFields` index
|
||||
* (name, description, every operation's name and description, every
|
||||
* trigger's name) - the exact same fields the original per-field search
|
||||
* over `Integration[]` matched against. See {@link IntegrationSummary} for
|
||||
* why this exists.
|
||||
*/
|
||||
export function toIntegrationSummary(integration: Integration): IntegrationSummary {
|
||||
const searchFields = [
|
||||
integration.name.toLowerCase(),
|
||||
integration.description.toLowerCase(),
|
||||
...integration.operations.flatMap((op) => [
|
||||
op.name.toLowerCase(),
|
||||
op.description.toLowerCase(),
|
||||
]),
|
||||
...integration.triggers.map((t) => t.name.toLowerCase()),
|
||||
]
|
||||
|
||||
return {
|
||||
type: integration.type,
|
||||
slug: integration.slug,
|
||||
name: integration.name,
|
||||
description: integration.description,
|
||||
bgColor: integration.bgColor,
|
||||
integrationType: integration.integrationType,
|
||||
searchFields,
|
||||
}
|
||||
}
|
||||
|
||||
export { blockTypeToIconMap } from '@/lib/integrations/icon-mapping'
|
||||
export {
|
||||
type OAuthServiceMatch,
|
||||
resolveOAuthServiceForIntegration,
|
||||
resolveOAuthServiceForSlug,
|
||||
} from '@/lib/integrations/oauth-service'
|
||||
export type { AuthType, FAQItem, Integration } from '@/lib/integrations/types'
|
||||
export type { AuthType, FAQItem, Integration, IntegrationSummary } from '@/lib/integrations/types'
|
||||
export { getAllBlockMeta, getBlockMeta, getTemplatesForBlock } from '@/blocks/registry'
|
||||
export type { BlockMeta, BlockTemplate } from '@/blocks/types'
|
||||
export { formatIntegrationType } from '@/blocks/types'
|
||||
|
||||
@@ -66,3 +66,28 @@ export interface Integration {
|
||||
/** Hand-authored landing content baked in at generation time (see `landing-content.ts`). */
|
||||
landingContent?: IntegrationLandingContent
|
||||
}
|
||||
|
||||
/**
|
||||
* The fields the `/integrations` catalog grid actually renders and searches
|
||||
* by, plus a precomputed, lowercased `searchFields` index (name, description,
|
||||
* every operation's name and description, every trigger's name) in place of
|
||||
* the full `operations`/`triggers` arrays. Shipping the full `Integration[]`
|
||||
* to that page's client component embeds every integration's complete field
|
||||
* set - including data the grid never renders (`tags`, `docsUrl`,
|
||||
* `landingContent`, ...) - in the initial HTML/RSC payload.
|
||||
*
|
||||
* `searchFields` stays an array, one entry per source field, rather than a
|
||||
* single joined string: matching must still require the query to fall
|
||||
* entirely within one field (as the original per-field `Integration[]`
|
||||
* search did), not span a field boundary a single concatenated string would
|
||||
* silently allow.
|
||||
*/
|
||||
export interface IntegrationSummary {
|
||||
type: Integration['type']
|
||||
slug: string
|
||||
name: Integration['name']
|
||||
description: Integration['description']
|
||||
bgColor: Integration['bgColor']
|
||||
integrationType: Integration['integrationType']
|
||||
searchFields: readonly string[]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user