Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarking

The Thesis

AI watermarking — embedding hidden signals in generated text, images, or audio so they can later be identified as machine-made — is being written into governance frameworks as if it were neutral infrastructure. This paper argues it is not. The statistical properties that make watermarks detectable vary with the content itself: the language, visual tradition, or demographic characteristics of the subject matter. That means a system designed to authenticate AI content might systematically fail — or over-flag — for non-English text, non-Western imagery, or certain demographic groups. The catch is that almost no published watermarking benchmark measures any of this. The authors are not presenting experimental proof of disparate impact; they are issuing a documented warning that the field is deploying authentication infrastructure without the bias auditing it would require of the AI systems it governs.

Catalyst

Watermarking mandates are moving from proposals to policy: the EU AI Act, the US Executive Order on AI, and platform governance frameworks now reference watermarking as a required provenance mechanism. That regulatory momentum means systems without equity audits are being locked into infrastructure before anyone has measured the gap. Simultaneously, large multilingual and multimodal generative models have expanded the content diversity that watermarking systems must handle — making the gap more consequential than it was when watermarking research focused on English text and Western photographic datasets.

What's New

Prior watermarking benchmarks — including the major ones reviewed here for text (such as work around Kirchenbauer et al.'s token-distribution method), image (SynthID-style perceptual embedding), and audio — evaluate detection accuracy on aggregate datasets that are overwhelmingly English, Western, and demographically narrow. Those benchmarks report a single headline accuracy number, which obscures whether performance degrades for underrepresented content types. This paper is the first systematic review to audit those benchmarks specifically for pluralistic coverage gaps, and it proposes three concrete evaluation dimensions — cross-lingual detection parity, culturally diverse content coverage, and demographic disaggregation — as a minimum standard for future work.

The Counter

This paper is a position paper, not an experimental study. The authors find that benchmarks lack disaggregated reporting — but absence of reported data is not evidence of disparate impact. It is equally possible that major watermarking systems perform comparably across languages and demographic content types, and that nobody has bothered to check because nothing suggested a problem. The theoretical argument — that watermark detectability depends on statistical properties of content, and those properties vary across cultures — is plausible, but plausible is not proved. The paper also conflates two distinct failure modes: a watermark that is harder to detect in Arabic text (a recall problem for provenance) and a watermark that over-flags non-watermarked Arabic text as AI-generated (a false-positive problem with civil-liberties implications). These require different fixes and different levels of urgency, and the paper treats them as a unified concern. Finally, the call to apply the same bias auditing standards to watermarking as to generative models sounds intuitive — but watermarking is a statistical signal, not a decision system, and the fairness frameworks developed for classifiers may not map cleanly onto it.

The Paper