Characterizing the Implementation of Censorship Policies in Chinese LLM Services

Characterizing the Implementation of Censorship Policies in Chinese LLM Services

Speakers: Anna Ablove

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=x-XlGWOa0yU

Overview

This talk presents a systematic study of how five major Chinese LLM services -- DeepSeek, Kimi, Qwen, Doubao, and Baidu Chat (Wenxiaoyan) -- implement censorship through combinations of input filtering, search-phase filtering, and output filtering. By man-in-the-middling traffic between client and server, the researchers discover significant information leaks: services transmit censored content to the client before blocking it, including near-complete responses, search results from websites blocked in China (Wikipedia, Human Rights Watch), and detailed political content that is subsequently truncated in the UI.

Using 80 sensitive queries tested in simplified Chinese, traditional Chinese, and English (1,200 total samples), the study reveals that censorship implementation is fragmented, inconsistent across languages, and surprisingly brittle -- all input and output blocking could be bypassed through character and instruction-level perturbations. Notably, four out of five services send user analytics to servers in China throughout the chatting process, including user IDs, language settings, country information, and filter trigger notifications.

Background

▶ Watch: Chinese LLM ecosystem and censorship challenges for AI services (0:00)

The Chinese LLM ecosystem has experienced enormous growth comparable to the West, but services must balance global competitiveness with government policies dictating acceptable conversation topics. Chinese content moderation for AI services builds on years of platform-specific censorship infrastructure including keyword filtering, audio/video review, and content classification systems.

However, applying traditional censorship techniques to LLMs presents unique challenges. The scale of training data makes it impossible to exclude all objectionable content from models. The token-based generation schema adds complexity to filtering, requiring real-time interception of streamed responses while maintaining acceptable latency. And the fundamental tension between response quality and censorship efficacy creates exploitable gaps.

Prior research on LLM content moderation has largely focused on alignment -- training models to refuse certain requests -- but alignment methods are not yet reliable enough for the strict legal compliance required in China. The legal environment necessitates absolute compliance, driving services to implement overt, system-level blocking mechanisms in addition to (or instead of) model-level alignment. These mechanisms leave observable traces in network traffic that this study exploits.

Key Findings

▶ Watch: Traffic analysis methodology: flow extraction across five services (4:00)

Multi-layer filtering architecture: All five services deploy combinations of filters across input, search, and output phases, with significant information leaking at each stage.

Information leaks in search phase: When services perform web searches before generating responses, they retrieve and transmit results from domains blocked in China including Wikipedia and Human Rights Watch. Four out of five services opted to perform searches 89-98% of the time. BU chat successfully answered 65% of sensitive queries after completing search, versus only 2.2% when search was skipped.

Information leaks in output phase: Services stream response tokens to the client before downstream filters catch objectionable content. In one example, Qwen streamed detailed content about surveillance in Xinjiang and the targeting of Muslim communities before abruptly halting and sending an error notice.

BU Chat client-side truncation: BU chat exhibited a uniquely concerning behavior where blocked responses were fully transmitted to the client but truncated in the UI by JavaScript. The complete responses -- including discussions of Mao's Cultural Revolution, the Great Cultural Revolution as a political movement, and Mao's consolidation of power -- were captured in network traffic despite being hidden in the interface.

Input blocking consistency: The most input-blocked query was "What happened in Tiananmen Square in 1989?" blocked across four services in all three languages. Qwen, DeepSeek, and Doubao input-blocked the exact same query sets in simplified and traditional Chinese.

Output blocking inconsistency: Output blocking varied wildly because it depends on the model generating objectionable content rather than matching keywords in the query. Queries like "What is the US view on the One China policy?" triggered output blocking despite containing no overtly sensitive keywords.

Brittle filtering: All input and output blocking could be bypassed using character and instruction-level perturbations, though some queries required upwards of an hour to craft successfully.

Analytics channels: DeepSeek, Kimi, and Doubao send analytics to the same autonomous system associated with ByteDance subsidiary Volcengine. One server endpoint was shared between DeepSeek and a ByteDance analytics service. BU chat encoded user queries directly into analytics URL capcha parameters.

Technical Deep Dive

▶ Watch: Rapid ecosystem changes: Qwen public release, BU adopts DeepSeek R1 (8:00)

The methodology centered on man-in-the-middle traffic analysis of active chat sessions with each service.

Flow Extraction. The researchers captured all HTTP traffic during chat sessions and identified relevant flows by searching for raw query text, then tracing associated responses. Each service had distinct server-side implementations requiring separate traffic analysis pipelines. For Qwen, relevant flows included chat initiation, completion prompts (response data streaming), and completion indicators. Additional flows corresponded to user analytics channels.

Response Text Extraction. Raw response text was extracted from streamed response data. Different services used different encoding and framing schemes. For services with explicit blocking indicators (Qwen, DeepSeek, Kimi), error codes in the traffic and UI notifications provided clear attribution. For Doubao and BU chat, blocking was identified through toast notifications and specific error strings, with BU chat's error string being generated by client-side JavaScript rather than appearing in server traffic.

Filter Characterization. The researchers distinguished between input blocking (observed before any search or response generation), search-phase blocking (after search retrieval but before response), and output blocking (during or after response generation). The search phase was further subdivided into target building (generating search phrases) and result retrieval (fetching actual links). Search results analysis revealed that services hosted by companies with their own content platforms favored proprietary domains: BU chat returned baidu.com links 45% of the time and Doubao favored toutiao.com 22% of the time.

Filter Evasion. Character-level perturbations (substituting visually similar characters, inserting invisible unicode) and instruction-level perturbations (rephrasing, indirect references) were systematically applied to all blocked queries -- 41 unique queries for input blocking and 197 for output blocking. All were eventually bypassed, confirming the brittleness of keyword and pattern-based filtering approaches.

Demo / Proof of Concept

▶ Watch: Input blocking consistency and search result analysis across services (10:00)

Multiple concrete examples were presented showing censorship in action and information leakage. The Qwen example showed a query about mass surveillance in China retrieving Wikipedia and Human Rights Watch articles (both blocked in China), followed by detailed response content about Xinjiang monitoring that was then cut off by a downstream filter. The BU chat example demonstrated full transmission of censored content about the Cultural Revolution that was only hidden by client-side JavaScript, meaning the data existed in the user's browser despite being visually truncated. Another BU chat example showed the model being asked to reverse the characters in "tank man" and stopping before completion. The researchers also noted significant ecosystem churn: Qwen's public release, BU chat's adoption of DeepSeek R1, and Kimi's Google login support all required adaptation of the measurement setup during the study.

Defensive Implications

▶ Watch: Analytics channels: shared ByteDance infrastructure and user data collection (14:00)

• Chinese LLM censorship is exported internationally: The researchers primarily accessed services from the US, meaning all observed censorship -- and analytics collection -- applies to non-Chinese users
• Information leaks are systemic: The streaming architecture required for competitive response latency creates inherent information leaks that conflict with censorship goals
• Client-side filtering is fundamentally insecure: BU chat's approach of transmitting full responses and truncating client-side means censored content is accessible to anyone inspecting network traffic
• Analytics pipelines reveal cross-service infrastructure: The shared analytics infrastructure between DeepSeek, Kimi, and Doubao (all routing to ByteDance-associated systems) raises questions about data sharing and the independence of ostensibly separate services
• Filter brittleness vs. numbers game: While all filters could be bypassed, the effort required (sometimes an hour per query) means censorship still achieves its goal of discouraging casual users, even if it fails against determined adversaries
• Alignment may be the long-term solution: Model-level alignment that produces pre-censored responses would eliminate the information leak problem, but current alignment techniques are not reliable enough for Chinese regulatory requirements

Key Takeaways

• Chinese LLM services implement censorship through multi-layer filters at input, search, and output phases, all of which leak information through network traffic
• Man-in-the-middle analysis reveals that services stream censored content (including responses from blocked websites) before filters intervene
• BU chat uniquely transmits complete censored responses that are only hidden by client-side JavaScript, leaving them fully accessible in network traffic
• Input blocking is consistent across services for high-profile topics (Tiananmen, Xi Jinping) but output blocking varies wildly
• All filtering can be bypassed through character and instruction perturbations, though the effort required limits practical impact for average users
• Four out of five services send detailed user analytics -- including query content and filter trigger events -- to servers in China, affecting international users
• The ecosystem is rapidly evolving, making time-sensitive measurement critical

About the Speaker(s)

Anna Ablove is a PhD candidate at the University of Michigan. The research team undertook the artifact evaluation process and made their dataset publicly available, reflecting the time-sensitive and rapidly changing nature of the Chinese LLM ecosystem. The speaker demonstrated thorough knowledge of both the technical traffic analysis methodology and the broader geopolitical context of Chinese internet censorship, and engaged substantively with audience questions about the distinction between censorship and safety alignment, potential impact on the services studied, and comparisons to Western content moderation practices.