Artificial intelligence (AI) is changing the world at a breathtaking pace, with tools capable of creating hyper-realistic images, videos, and even conversations. While this technology holds tremendous potential, it also has a dark side, one that’s quickly become a major concern for governments around the globe. In response, China has taken a decisive step, unveiling strict new regulations that demand all AI-generated content be clearly labeled. But this regulation, while seemingly aimed at protecting the public from misinformation, raises serious questions about freedom, privacy, and the possibility of state overreach.

What Are AI Watermarks, and How Are They Implemented?

AI watermarks are essentially digital labels that mark content created by artificial intelligence. These watermarks can be both explicit, like a visual tag or audio notification, and implicit, like hidden metadata that the average user can’t see. China’s latest regulation, introduced by the Cyberspace Administration on September 14, takes a comprehensive approach to these watermarks, requiring a variety of labeling strategies to ensure that AI-generated content is immediately recognizable.

Explicit watermarks are relatively straightforward. They could include visible marks on images, notifications at the start of AI-generated videos, or even Morse code audio signals that indicate AI involvement. These visible signs make it clear to users that they are interacting with content created by artificial intelligence. As the draft regulation puts it, these watermarks should be “conspicuous” so that viewers are never in doubt about the content’s origins.

However, explicit labels can easily be tampered with. Someone could crop a watermark out of an image or cut a video clip short before the AI notification plays. To address this, China’s regulation requires companies to also embed implicit labels into the metadata of the files. Metadata is the hidden information stored with digital content, and in this case, it would include tags like “AIGC” (Artificially Intelligent Generated Content) along with encrypted data about who produced and distributed the content. This means that even if a user tried to edit out the explicit markers, the AI label would still be embedded within the file itself.

The regulation goes even further, suggesting that companies add invisible watermarks—labels that are woven into the content in such a way that they’re nearly impossible to detect with the naked eye. This way, even if people don’t realize they’re being tracked, the government and platforms would still have the capability to identify content origins. According to Sam Gregory, executive director of Witness, a human rights organization, these requirements present “technical challenges and costs,” suggesting that achieving compliance across different platforms and AI tools is a daunting task.

The Problem China Says It Wants to Solve

The stated goal of these new regulations is to combat misinformation and fraud. As AI technology advances, the potential for misuse grows, from fake news and manipulated videos to more sinister applications like deepfake pornography or AI-generated scams. China, with a population of over a billion internet users, has already experienced the impact of these misuses. Fraud linked to AI generated over 185 million yuan (roughly $26 million) in losses during the first half of 2024—an alarming increase from previous years.

In response to such threats, China’s regulations aim to create a “healthy cyber information ecosystem.” By enforcing strict labeling of all AI-generated content, the government hopes to make it more difficult for bad actors to deceive the public. The guidelines require AI content providers—including social media platforms, tech companies, and even individuals using AI tools—to clearly mark anything produced by artificial intelligence. For AI videos, for instance, labels would need to appear at the beginning, end, and even at “appropriate” times throughout the video.

These regulations don’t just apply to companies that create AI content; they also put responsibility on platforms where AI content is shared. According to the draft, platforms must “examine shared files for implicit labels and AI-generation traces.” If metadata reveals the content is AI-generated, or if the uploader discloses it, the platform must add a label. If the platform even suspects that the content was made using AI, it must label it as such. This responsibility effectively turns social media and content-sharing platforms into enforcers of the state’s new rules, and failure to comply could have severe consequences.

But even if China frames these measures as a necessity to combat fraud, there’s a growing suspicion that this is about more than just misinformation. With the power to track the origin of any AI content, the government gains an unprecedented level of control over what people create and share.

What About Freedom of Expression and Privacy?

There’s a darker side to this new regulation, one that has civil rights advocates deeply worried. If China’s government can label and track every piece of AI-generated content, it’s not hard to imagine how this power might be misused. Implicit labels and metadata give authorities the ability to trace content back to its origin, identifying who created it and when. In a country known for its strict censorship and surveillance practices, this raises alarm bells about how these tools could be used to stifle dissent or control speech.

Sam Gregory from Witness sums up the concern, stating, “The big underlying human rights challenge is to be sure that these approaches don’t further compromise privacy or free expression.” He warns that while AI labels and watermarks might help identify misinformation, they also provide the state with sophisticated tools to suppress freedom of expression. If someone posts content critical of the government, authorities could easily use these embedded labels to trace it back to the creator, increasing the risks for individuals who wish to speak out.

This is especially troubling in light of China’s history of censorship and internet control. Under this new regulation, platforms like Douyin (China’s version of TikTok) and WeChat will have to police every single piece of user-uploaded content for AI involvement. Jay Si, a Shanghai-based lawyer, notes that “If WeChat or Douyin needs to examine every single photo uploaded to the platform and check if they are generated by AI, that will become a huge burden in terms of workload and technical capabilities for the company.” But beyond the technical challenges, it raises serious concerns about the chilling effect this could have on free speech. Users might hesitate to post anything remotely critical, knowing that every post is subject to examination for AI involvement and traceability.

A Bid for Control in the Global AI Race

China is not the only country attempting to regulate AI-generated content. The European Union’s AI Act and recent legislation in California also require AI content labeling. However, China’s approach is notably more extreme, especially in its enforcement mechanisms. The regulation makes it explicitly clear that platforms are not only responsible for labeling AI content but are also liable if they fail to do so. In the United States, platforms are generally not held responsible for the content uploaded by users. This stark difference highlights how China is using AI regulations not just as a protective measure but as a tool of control.

Jeffrey Ding, an assistant professor at George Washington University, points out that China’s policymakers are inspired by global trends but often take a more authoritarian approach. He notes, “Chinese policymakers and scholars have said that they’ve drawn on the EU’s Acts as inspiration for things in the past.” However, unlike in Europe or the United States, where the focus is on transparency and user choice, China’s regulation is about state control and compliance.

Chinese authorities have made it clear that they want to set the standards for AI regulation globally. Angela Zhang, a professor studying Chinese tech regulations, says that “China is definitely ahead of both the EU and the United States in content moderation of AI,” largely because of its desire to control how technology aligns with government policies. The country’s quick and decisive move to regulate AI-generated content, even before most Western nations, is part of a larger ambition to dominate global AI standards and ensure that technology does not threaten the state’s grip on power.

Balancing Innovation and Control

Despite China’s determination to regulate AI, there is still pushback from within the industry. AI companies have voiced concerns about the increased costs and limitations that come with compliance. Earlier versions of AI regulations were significantly watered down after feedback from the industry, indicating that even in China, where government control is extensive, the need for technological growth and innovation cannot be entirely ignored.

“What we’ve seen is the Chinese government really trying to walk this fine tightrope between ‘making sure we maintain content control’ but also ‘letting these AI labs in a strategic space have the freedom to innovate,’” says Jeffrey Ding. There’s a delicate balance between ensuring that AI isn’t used to spread harmful misinformation and making sure that regulation doesn’t stifle the very industry China wants to lead globally.

For now, these regulations are still in draft form, open to public comment until October. But considering China’s history of pushing through its policies with minimal changes, it seems likely that AI companies and platforms will soon face a new era of heavy-handed oversight.

The draft regulation also hints at what could be a burgeoning black market for AI services that avoid the costly compliance measures. As Sima Huapeng, founder and CEO of a Chinese AI company, points out, “If a feature is optional, then most likely companies won’t add it to their products. But if it becomes compulsory by law, then everyone has to implement it.” While legitimate companies may comply, there’s a risk that underground services will flourish, offering unregulated, cheaper alternatives that bypass these stringent rules.

Conclusion: Protection or a Path to Total Control?

China’s push for AI watermarks is framed as a way to protect citizens from misinformation and fraud. On the surface, it’s a move that seems logical—after all, the misuse of AI has led to real harm, from scams to fake news. But beneath this veneer of protection lies a far more troubling reality. The ability to track and label every piece of AI content grants the government extraordinary power—not just over the technology itself, but over the people using it.

As the world watches China lead the charge in regulating AI, the question remains: are these measures really about protecting people, or are they about protecting power? The answer may very well determine how AI shapes our future—for better or for worse.

ACZ Editor: A watermark implies tracking. If you can track an article back to the AI source, then the next regulation will be to maintain a track back to the person who requested it. The CCP has a habit of tracking an punishing people who say things they don’t like. And of course the companies running AI would either be forced to report anyone producing information the government doesn’t like, or could force them to only allow the AI to say things complimentary to the government (which it has already done in at least one case). And of course, Chinese authorities can accuse anyone they like of using AI and then cutting out the watermarks.

This is a another step toward absolute and instant information control in China.