Identification and Anonymization of Named Entities in Unstructured Information Sources for Use in Social Engineering Detection
A new pipeline scrapes Telegram for cybercrime data, transcribes audio, and anonymizes personal details — letting researchers study social engineering without breaking privacy law.
The Thesis
Cybersecurity researchers studying social engineering — manipulation tactics used to trick people into giving up passwords or money — are stuck: the best training data lives on criminal platforms like Telegram, but collecting it without stripping personal information violates GDPR and similar laws. This paper proposes a full pipeline that harvests Telegram messages, images, and audio, transcribes speech to text using a model called Parakeet, and then runs Named Entity Recognition (NER) — software that spots names, phone numbers, and other identifying details — to anonymize the data before it enters a research dataset. The authors claim their custom transformer-based NER models (transformer meaning a neural network architecture that reads text in context, like the engine inside ChatGPT) outperform Microsoft's off-the-shelf Presidio tool on detecting sensitive fields. The practical payoff is a legally defensible way to build labeled datasets for training fraud and social-engineering detectors. The catch is that the paper is primarily a system description: the datasets and downstream detection models are not yet published.
Catalyst
GDPR enforcement has intensified since 2022, with multi-million-euro fines making institutions genuinely afraid to handle raw personal data in research pipelines. Simultaneously, Telegram has become a documented hub for fraud coordination, making it a high-value target for cybercrime researchers who previously had no compliant way to use that data. Advances in open-weight speech-to-text models — including NVIDIA's Parakeet — now make audio transcription accurate enough to include in a privacy pipeline without a separate costly annotation step.
What's New
Prior NER-based anonymization work, including Microsoft Presidio, relied on rule-based matching and general-purpose pretrained models not fine-tuned for cybercrime language. Those tools miss slang, code-switching, and the informal register common in criminal Telegram channels. This paper fine-tunes transformer models specifically on that domain and adds speech-to-text as a preprocessing step, extending anonymization to audio messages — a gap earlier systems left entirely unaddressed.
The Counter
This paper describes a pipeline but does not release the dataset, the fine-tuned models, or reproducible benchmarks that outside researchers can verify — that is a significant credibility gap. The F1-score comparisons against Presidio are narrow in scope: Presidio is a general-purpose tool not designed for criminal-slang text, so beating it is a low bar. The anonymization metrics the authors introduce to measure 'structural coherence' are novel and self-defined, meaning there is no established community standard to validate them against. Perhaps most importantly, the paper does not demonstrate that the anonymized dataset actually improves a downstream social-engineering detector — the whole stated goal — so the end-to-end value remains unproven. Telegram's API terms and the platform's own moderation changes could also break data collection at any time, making the pipeline brittle in practice.
Longs
- PANW (Palo Alto Networks) — threat intelligence platforms that ingest social-engineering signals
- CRWD (CrowdStrike) — detection models trained on richer, compliant cybercrime datasets
- MSFT (Microsoft) — Presidio is their open-source tool; competitive pressure may accelerate its improvement
- CIBR (cybersecurity ETF) — broad exposure to fraud detection and threat intelligence vendors
Shorts
- Microsoft Presidio (as a standalone product) — the paper demonstrates domain-fine-tuned models outperform it, pressuring its adoption in compliance-sensitive research contexts
- Vendors selling raw, non-anonymized threat-intelligence feeds — a compliant open-source pipeline undercuts the premium for manually curated, scrubbed datasets
Enablers (Picks & Shovels)
- NVIDIA NeMo / Parakeet — the speech-to-text model identified as best-performing in the paper
- Microsoft Presidio — open-source NER baseline the authors benchmark against
- Hugging Face Transformers — the ecosystem used to fine-tune the custom NER models
- Telegram Bot API — the data collection layer the pipeline depends on
Private Watchlist
- Abnormal Security — email social-engineering detection, would benefit from richer training data
- Hiya — voice fraud detection, relevance to audio anonymization pipeline
- Graphika — open-source intelligence on criminal networks, adjacent compliance problem
Resources
The Paper
This study addresses the challenge of creating datasets for cybercrime analysis while complying with the requirements of regulations such as the General Data Protection Regulation (GDPR) and Organic Law 10/1995 of the Penal Code. To this end, a system is proposed for collecting information from the Telegram platform, including text, audio, and images; the implementation of speech-to-text transcription models incorporating signal enhancement techniques; and the evaluation of different Named Entity Recognition (NER) solutions, including Microsoft Presidio and AI models designed using a transformer-based architecture. Experimental results indicate that Parakeet achieves the best performance in audio transcription, while the proposed NER solutions achieve the highest f1-score values in detecting sensitive information. In addition, anonymization metrics are presented that allow evaluation of the preservation of structural coherence in the data, while simultaneously guaranteeing the protection of personal information and supporting cybersecurity research within the current legal framework.