mirror of
https://github.com/xming521/WeClone.git
synced 2026-09-01 15:36:15 +08:00
xming521
a24442b78d
Improves PII detection for Chinese language data by: - Adding a Chinese-specific PII detector that inherits from the base PII detector. - Incorporating custom recognizers for common Chinese PII patterns (e.g., phone numbers, ID cards). - Filtering out irrelevant country-specific entities when processing Chinese text. - Adding unicode escape id recognizer. These changes ensure more accurate and comprehensive identification of sensitive information in Chinese datasets.