mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-01 14:59:19 +08:00
Theodore Li
0191a614b6
* feat(presidio): build & own combined analyzer+anonymizer image Replace the stock mcr.microsoft.com/presidio-* sidecar images with a single image we build and push to ECR/GHCR. A thin FastAPI service constructs one AnalyzerEngine + one AnonymizerEngine at startup and serves both on port 3000 (/health, /supportedentities, /analyze, /anonymize) so the app needs one PRESIDIO_URL. English only; pinned presidio 2.2.362 + en_core_web_lg 3.8.0. Bakes in the native check-digit VIN recognizer and registers 12 English recognizers Presidio ships but does not load by default (UK_NINO, AU_*, IN_*, SG_*), taking the supported English set from 19 to 32. * feat(presidio): add multi-language support (es/it/pl/fi) Configure a multi-language spaCy NLP engine (en/es/it/pl/fi lg models) and explicitly register the national-id recognizers Presidio ships but does not load by default: ES_NIF/NIE, IT_FISCAL_CODE/DRIVER_LICENSE/VAT_CODE/PASSPORT/ IDENTITY_CARD, PL_PESEL, FI_PERSONAL_IDENTITY_CODE. Verified the NLP-engine + explicit-registration path detects in-language (Finnish id, score 1.0). * improvement(presidio): address review feedback - Register VIN under all served languages, not just en (Bugbot: VIN missed for non-English language routing). - Bump HEALTHCHECK start-period to 180s — five lg models load at import (Bugbot). - Drop --no-cache-dir so the pip cache mount actually works (Greptile). - Pydantic request models for /analyze + /anonymize so missing 'text' returns 422 not 500; default operator 'type' to 'replace' instead of KeyError->500 (Greptile). * refactor(pii): rename presidio image artifacts to pii Rename the image/repo/secret/files from 'presidio' to 'pii' for clarity — the service does PII detection + anonymization (and backs the guardrails block's block/mask), not just redaction, and 'pii' matches existing pii-* naming. docker/presidio.Dockerfile -> docker/pii.Dockerfile docker/presidio/ -> docker/pii/ ghcr.io/simstudioai/presidio -> .../pii ECR_PRESIDIO secret -> ECR_PII (infra side already renamed) No behavior change — paths/identifiers only. * refactor(pii): move service to apps/pii, make image ECR-only - Move server.py + requirements.txt from docker/pii/ to apps/pii/ (source belongs under apps/, matching app/realtime; Dockerfile stays in docker/). Add a minimal @sim/pii package.json so the apps/* bun workspace glob accepts the Python service. - Repoint docker/pii.Dockerfile COPY paths to apps/pii/; rename the container user presidio -> pii. - Drop GHCR for pii — it's a private ECS sidecar pulled from ECR, never published. Removed it from the arm64/manifest (GHCR-only) jobs and guarded the build-amd64 tag step to skip GHCR when no ghcr_image is set.