China addresses the misuse of face-swapping AI with new regulation - PingWest

China addresses the misuse of face-swapping AI with new regulation

China addresses the misuse of face-swapping AI with new regulation

13 hours ago

The Cyberspace Administration of China, the Ministry of Industry and Information Technology and the Ministry of Public Security has jointly issued a "Regulations on the Management of Internet Information Service Depth Synthesis" (hereinafter referred to as "Regulations"), which would come into force from January 10, 2023.  

According to the regulation, services that provide intelligent dialogue, synthetic human voice, synthetic face generation, immersive synthetic scene generation and other functions that generate or significantly change the content of information shall be noticeably marked to avoid public confusion or misidentification. Any organization and individual shall not use technical means to delete, tamper with, or conceal the relevant marks.

In short, the regulation asks users of face-swapping AIs and similar services to put a noticeable watermark to clarify the nature of the generated video, audio or image, and neither the users nor any third party shall remove the watermark.  

The Regulation also asks technology provider of such services to shoulder some of the responsibility, including authentication of users' identity, managing and monitoring the synthetic content, establishing a mechanism for people to file complaints about untruthful content generated by such technology.

This is a step up from China’s previous regulation regarding face-swapping AI. In November 2019, the Cyberspace Administration of China, the Ministry of Culture and Tourism and the General Administration of Radio, Film and Television published a “Internet Audio and Video Information Services Management Regulation.” The 2019 document was the first time Chinese regulation address the  face-swapping technology to crackdown misinformation, yet the language was rather vague — “corresponding measures shall be taken to in a timely manner to dispute the misinformation” caused by “the production, publication and use of false images, audio and video generation technology based on deep learning, virtual reality, etc.”

Face-swapping, voice-generating or video generation AIs, such as the infamous “DeepFake”, ”DeepNude” and its Chinese counterpart “ZAO”, have become a major cybersecurity concern. Such technology, when in the wrong hands, can be used to create graphic pornography without consent or conduct financial crimes.

A Netherland-based cybersecurity company, Deeptace, once published a report after reviewing 14,678 videos and concluded that 96% of “deepfake videos” are porn and a slew of porn sites dedicated to face-swapping videos have popped up. In addition, In 2019, an U.K. based energy company was scammed out of 220,000 Euros when a caller used AI technology to present as an executive of the company.