As such, one of the most important points stressed in recent MERICS and Trivium reports is that as yet there is no integration between ratings systems and security technologies like facial recognition or internet monitoring.
Indeed, China’s major tech companies have so far been largely shut out from its development.
However, the proximity of the SoCS to the security apparatus and the core role Xi Jinping envisions for the system alongside Internet+Monitoring and Internet+Governance policies should encourage vigilance.
Within the highly centralised and opaque security and surveillance apparatus, there is considerable appetite for the deployment of artificial intelligence, machine learning and big data analysis.
These technologies tend to fall under the category of “dual-use”, meaning their capabilities could be deployed both within a civil/commercial setting or used in more oppressive ways that risk human rights violations.
Their extensive use as part of a security dragnet against Uighur Muslims in Xinjiang is paradigmatic of the problem.
It’s within these initiatives that many of China’s tech unicorns have cut their teeth and risen to the forefront of the global digital technology market.
According to a policy brief released last year by the US Center for Security and Emerging Technology, since 2008, at least 80 countries have adopted Chinese surveillance technology platforms.
These platforms are more than just security cameras: they integrate multiple government databases and provide analytic capabilities that can support multiple command and control centres.
The past two decades have seen China’s Ministry for Public Safety (MPS) partner with various tech and digital research agencies to develop their surveillance systems. The best known of these involving the vast national network of CCTV cameras and other so-called public safety measures are Golden Shield (金盾工程), Safe Cities (平安城市) and SkyNet (天网工程).
Sharp Eyes (雪亮工程) (a term originating during the Cultural Revolution) is the latest iteration of this policy of total surveillance which seeks to integrate and upgrade the network to include civilian-owned cameras, with special emphasis on rural areas which have so far escaped the net.
Despite their undeniable scope, the omniscience of China’s facial recognition systems, which state-run media frequently hype, were delivered a blow in 2018 when an executive from Megvii – the AI company which created the Face++ ID system – claimed it could neither run 24/7 nor scan more than 1000 faces at a time.
While the SoCS remains comparatively data-poor where citizens are concerned, this is not the case for Police Cloud (警务云), another MPS initiative that aggregates a vast amount of personal data from public and private sources, including medical records, IP addresses, social media usernames and some biometric data.
It’s these systems that have been most prominent during the years-long security operation in China’s Northwest, which this month a UK-based tribunal concluded amounted to genocide against the Uighur community.
Though data-driven authoritarianism has played a role, the brutality directed at the Uighurs, including the mass internment of up to a million people in re-education camps, is a distinctly crude form of state violence in which social credit has no discernible role.
Governing Between Order and Chaos
The ending of the original five-year construction plan for the Social Credit System coincides with a major upheaval in the legal environment around China’s rapidly developing digital technology sector.
The atmosphere of toleration which had seen vast growth and innovation over the preceding decade has shifted in the last twelve months to one of direct confrontation. October 2020 saw the start of Operation Cyber Sword (网剑行动), a wide-ranging regulatory clean-up of China’s digital economy, ironically using the SoCS as its principal weapon.
According to another MERICS report, fines against hundreds of companies have been issued totalling over $3 billion, products have been removed from digital stores, while Alibaba Group’s Jack Ma – perhaps more than anyone symbolic of China’s recent tech success – vanished from public view for three months.
The report concludes that this crackdown represents a consolidation of CCP institutional power and the implementation of a broad legal framework in line with another recently announced ambitious roadmap: the Plan for the Construction of Rule of Law in China (2020-2025), which sets the legal foundation for Xi Jinping’s vision for data-driven governance, and which includes a prominent place for the SoCS.
To understand the meaning of these recent policy drives it’s important to note that the Chinese government’s understanding of the Rule of Law departs significantly from internationally recognised norms insofar as they explicitly reject core tenets such as separation of powers or an independent judiciary.
A recent explainer on ‘Xi Jinping Thought on the Rule of Law’ by the German Institute for International and Security Affairs emphasised that Fǎzhì (法治), the word which in CCP documents is often translated as Rule of Law, would be better understood as rule BY law.
This would capture the CCP leadership’s approach to law as an instrument of rule rather than having any intrinsic value itself and is in line with longstanding legalistic traditions in China which demands the strict application of rules by a strong centralised state.
Just as Beijing has placed limits on the degree of automation within the social credit systems, there are also signs that it wants to set boundaries to the use of AI within China.
In July this year, a PRC think tank under China’s Ministry of Industry and Information Technology (MIIT) authored a white paper outlining their approach to the development of what they termed “Trustworthy Artificial Intelligence”.
The paper emphasised the need to strengthen judgement and watchfulness to ensure the stability, explainability and fairness of future AI, in line with the expectations of the people and national security.
Taken together, these regulatory actions, plans and policy initiatives amount to a coordinated effort to both extend and control the use of digital technologies within Chinese society and government, the scope of which has no parallel in the West, where market-led development and self-regulation are largely the rule.
Considering the historical laissez-faire attitude to the development of digital technologies in Europe and the US, fears over the imposition of a “Chinese style authoritarian digital surveillance system” perhaps reflect anxieties about the reach and potential misuse of technologies already embedded throughout Western democracies, and most recently their use during the pandemic.
Measures designed to control the transmission of Coronavirus have undoubtedly accelerated processes of “datafication” and “dataveillance”, the basis upon which any social credit system would function.
Credit ratings and comparison systems of many kinds are already ubiquitous throughout Europe and the US, from financial credit reporting agencies, to travel comparison services like Tripadvisor and even dating apps. These companies hold a vast amount of data on the citizens of democratic countries.
The three largest credit reporting agencies (Experian, TransUnion, and Equifax) together hold data for over two billion individuals and businesses. Each one is developing Big Data and artificial intelligence technologies to automate their assessment systems, and any one out of Amazon, Google, Facebook and Apple hold more personal data on more individuals than the Chinese Communist Party could dream of.
What is currently lacking is central government coordination of databases and the kind of unified vision for digital tech that China is pioneering. That may be changing as the economic potential of Big Data and AI becomes more attractive as a way of boosting growth.
In the UK a recent government report on the future of citizen data concluded that the collecting and sharing of such data could benefit the economy by “boosting productivity and trade, enabling innovative products, improving public service delivery and informing scientific research”.
It’s striking that the question of limits to digital technologies and their socially disruptive effects are largely absent from the discussion, despite rising public discontent over the harms of social media algorithms and their role in fostering political division and the rise of “post-truth” culture.
One interpretation of China’s attempt to contain the development of digital technologies within a centralised law-based framework is that they are trying to avoid the same fate.
The dream of an algorithmically governed society, harmonised at a granular level by AI and Big Data systems, is largely a vision of Silicon Valley, synonymous with the abolition of traditional political authority, and even the notion of sovereignty itself.
But, if decision making escapes human cognitive limitations into artificial intelligence, will there be a place in the future for the politician or party apparatchik?
This is perhaps a more pressing question for the Chinese Communist Party, for whom a centralised – and up until now – still personalised bureaucratic presence is inseparable from their form of government.
It would be a mistake to assume that forces driving the transformation of Western political systems and Western societies may not also impact Chinese society and government in equally disruptive ways.
How ‘Xi Jinping Thought’ might integrate a depersonalised algorithmically governed China is a question fraught with tensions.
Unlike Europe and America, China, it seems, is taking these tensions seriously.
Pandemic Politics Anxieties in the first article in this series. Photograph courtesy of Joel Schalit. All rights reserved.
