Files
sim/apps/docs/app/sitemap.ts
T
Emir Karabeg 5274efd8f9 improvement(seo): optimize sitemaps, robots.txt, and core web vitals across sim and docs (#4170)
* improvement(seo): optimize sitemaps and robots.txt across sim and docs

- Add missing pages to sim sitemap: blog author pages, academy catalog and course pages
- Fix 6x duplicate URL bug in docs sitemap by deduplicating with source.getLanguages()
- Convert docs sitemap from route handler to Next.js metadata convention with native hreflang
- Add x-default hreflang alternate for docs multi-language pages
- Remove changeFrequency and priority fields (Google ignores both)
- Fix inaccurate lastModified timestamps — derive from real content dates, omit when unknown
- Consolidate 20+ redundant per-bot robots rules into single wildcard entry
- Add /form/ and /credential-account/ to sim robots disallow list
- Reference image sitemap in sim robots.txt
- Remove deprecated host directive from sim robots
- Move disallow rules before allow in docs robots for crawler compatibility
- Extract hardcoded docs baseUrl to env variable with production fallback

* fix(seo): remove homepage new Date(), guard latestModelDate empty array

* improvement(seo): consolidate DOCS_BASE_URL, optimize core web vitals

Extract hardcoded https://docs.sim.ai into shared DOCS_BASE_URL constant
in lib/urls.ts and replace all 20+ instances across layouts, metadata,
structured data, LLM manifest, sitemap, and robots files. Remove
OneDollarStats analytics script and tighten CSP for improved core web vitals.

* fix: removed onedollarstats from bun lock

* fix(seo): guard per-provider Math.max, consolidate docs robots to single wildcard
2026-04-15 12:13:30 -07:00

43 lines
1.1 KiB
TypeScript

import type { MetadataRoute } from 'next'
import { i18n } from '@/lib/i18n'
import { source } from '@/lib/source'
import { DOCS_BASE_URL } from '@/lib/urls'
export const revalidate = 3600
export default function sitemap(): MetadataRoute.Sitemap {
const baseUrl = DOCS_BASE_URL
const languages = source.getLanguages()
const pagesBySlug = new Map<string, Map<string, string>>()
for (const { language, pages } of languages) {
for (const page of pages) {
const key = page.slugs.join('/')
if (!pagesBySlug.has(key)) {
pagesBySlug.set(key, new Map())
}
pagesBySlug.get(key)!.set(language, `${baseUrl}${page.url}`)
}
}
const entries: MetadataRoute.Sitemap = []
for (const [, localeMap] of pagesBySlug) {
const defaultUrl = localeMap.get(i18n.defaultLanguage)
if (!defaultUrl) continue
const langAlternates: Record<string, string> = {}
for (const [lang, url] of localeMap) {
langAlternates[lang] = url
}
langAlternates['x-default'] = defaultUrl
entries.push({
url: defaultUrl,
alternates: { languages: langAlternates },
})
}
return entries
}