mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-17 17:18:28 +08:00
Theodore LiandClaude Fable 5
4e6594dc54
* feat(pii): add opt-in GLiNER NER engine (PII_ENGINE), device-agnostic Swap the 4 NER entity types (PERSON/LOCATION/NRP/DATE_TIME) to a single multilingual GLiNER zero-shot model when PII_ENGINE=gliner; spaCy stays the default and all ~36 regex/checksum recognizers are identical on both engines. Device-agnostic via PII_DEVICE / cuda auto-detect — same code on Fargate CPU now and EC2-GPU later. - engines.py: side-effect-free builders; SharedModelGLiNERRecognizer loads ONE model shared across the 5 per-language instances and restricts labels to the entities it owns; small spaCy models keep tokenization/lemmas for the regex recognizers; fail-fast on the lean image - pii.Dockerfile: multi-stage — default target unchanged (lean spaCy); --target gliner is a superset (torch CPU + gliner + baked model) where both engines work; gliner-gpu scaffold for the GPU fleet - CI publishes the gliner variant (:staging-gliner/:latest-gliner, amd64) - Helm: pii.engine / pii.device values wired to PII_ENGINE/PII_DEVICE - scripts/bench_engines.py: throughput + NER-parity diff harness - tests: unit (mocked GLiNER) + in-image integration for both engines Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Up3F97mjCH9HCj1pX4J8VJ * refactor(pii): ship both engines in one image — engine is a pure env flip Collapse the gliner build target into the single pii image: spaCy lg models, torch (CPU), gliner, and the baked GLiNER weights all ship in it, so PII_ENGINE switches engines with no image swap and no tag matrix. CI reverts to the single pii build (no -gliner tags). The GPU variant becomes the same Dockerfile built with --build-arg TORCH_INDEX_URL=.../cu128. Image grows ~6.1GB -> ~9.6GB. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Up3F97mjCH9HCj1pX4J8VJ --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
103 lines
1.3 KiB
Plaintext
103 lines
1.3 KiB
Plaintext
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
|
|
|
|
# dependencies
|
|
/node_modules
|
|
/apps/**/node_modules
|
|
/packages/**/node_modules
|
|
/scripts/node_modules
|
|
|
|
# bun specific
|
|
bun-debug.log*
|
|
|
|
# cursor debug logs
|
|
.cursor/debug-*.log
|
|
|
|
# this repo uses bun.lock; package-lock.json files are accidental
|
|
package-lock.json
|
|
|
|
# testing
|
|
/coverage
|
|
/apps/**/coverage
|
|
|
|
# next.js
|
|
/.next/
|
|
/apps/**/out/
|
|
/apps/**/.next/
|
|
/apps/**/build
|
|
|
|
# production
|
|
/build
|
|
/dist
|
|
**/dist/
|
|
**/standalone/
|
|
sim-standalone.tar.gz
|
|
|
|
# redis
|
|
dump.rdb
|
|
|
|
# misc
|
|
.DS_Store
|
|
*.pem
|
|
|
|
# env files
|
|
.env
|
|
*.env
|
|
.env.local
|
|
.env.development
|
|
.env.test
|
|
.env.production
|
|
|
|
# editor swap files
|
|
*.swp
|
|
*.swo
|
|
*.swn
|
|
|
|
# vercel
|
|
.vercel
|
|
|
|
# typescript
|
|
*.tsbuildinfo
|
|
next-env.d.ts
|
|
|
|
# cursorrules
|
|
# .cursorrules
|
|
|
|
# docs
|
|
/apps/docs/.source
|
|
/apps/docs/.contentlayer
|
|
/apps/docs/.content-collections
|
|
|
|
# database instantiation
|
|
**/postgres_data/
|
|
|
|
# collector configuration
|
|
collector-config.yaml
|
|
docker-compose.collector.yml
|
|
start-collector.sh
|
|
|
|
# Turborepo
|
|
.turbo
|
|
|
|
# VSCode
|
|
.vscode
|
|
|
|
# IntelliJ
|
|
.idea
|
|
|
|
## Helm Chart Tests
|
|
helm/sim/test
|
|
i18n.cache
|
|
|
|
## Claude Code
|
|
.claude/launch.json
|
|
.claude/worktrees/
|
|
.claude/scheduled_tasks.lock
|
|
.deepsec/
|
|
|
|
# Personal Cursor Skills
|
|
.cursor/skills/ask-sim/
|
|
|
|
# Python (apps/pii tests/tooling)
|
|
__pycache__/
|
|
.pytest_cache/
|