Big Data Ethics: What does China's social credit system mean for the West?

Social credit scores in China blur boundaries between commerce, state and private life

China recently announced it plans to implement a nation-wide social credit system by 2020.  The system, already being piloted in some regions, gathers data from smartphones, facial recognition cameras and other AI technology.  It then feeds this data into an algorithm that assigns each user a social credit score.  A person's social credit score will affect everything from their mortgage rate to their internet speed.

The main corporations currently piloting the project are the popular online payment platforms Alipay and WeChat Pay.  As the Financial Times points out, there’s a clear conflict of interest in an online payment platform giving out social credit scores.  Cash is quickly becoming obsolete in China, with smartphone apps like Alipay and WeChat Pay being used by more and more people to pay for everyday tasks like buying groceries, paying at restaurants, booking doctor’s appointments and paying for parking.  The apps are so pervasive that when an “untrustworthy” individual is blocked from using these platforms, it has a crippling effect on their day-to-day life.

And while we have no choice but to begrudgingly trust large corporations with our purchase history, do we necessarily want them to send that information to the government?  With this kind of data available, people’s behavior as consumers becomes inextricably linked to their behavior as citizens.  The Economist warns that in the worst-case scenario, the social credit system could become a “360-degree digital-surveillance panopticon.”

But that’s only China.  Should we be worried?

Chinese people might be more married to their phones than we are in the West, but the pervasive Internet of Things is slowly but surely creeping into our lives as well.  According to the same Economist article, the NSA is capable of collecting as many as 42 billion internet records every month – and 5 billion cell phone location records in just a day.  Data-hungry devices such as Fitbits, smart Barbies and Alexa & company are steadily gathering data on an increasing portion of our lives, and Facebook has already secured a patent that enables lenders to determine a person’s credit score based partly on their friends’ financial history.  There’s even precedent for an eerily similar social credit rating system based in the US.

When you start putting everything together, you realize corporations and governments actually know quite a lot about you.  And while you may shrug this off as irrelevant to you in your day-to-day life, it’s a fact that data now contributes more to worldwide GPR than tangible goods.  The concept of mining this information and giving it real-world consequences is not too far away – for example, some companies are already offering health insurance discounts for people who wear a fitness tracker.  As we jump at these small incentives like moths to a flame, enthusiastically adopting privacy-invading IoT devices left and right, governments and corporations wait calmly for us to play straight into their hands.

The social credit system doesn’t need any help – it will implement itself

For now, China’s social credit system touts itself as a game, with most people eager to join and to parrot the advantages of having “control” over their own social credit scores.  It’s an ingenious take on the brain’s behavioral learning system, playing on our natural tendency to conform with whatever gives us a reward.  The proof: the social credit system started out as a voluntary incentive, and was eagerly received by millions of people.  As the data piles up, the state’s gaze becomes more and more omniscient, while the algorithms used to make weighty decisions regarding people’s livelihoods remain in opacity.

Aided and abetted by the shear reams of data collected by giant corporations, the social credit system has catapulted Orwell’s Big Brother theories neatly into the 21st century.  It’s the perfect marriage between corporate data collection and state-run surveillance, a step so obvious we could kick ourselves for not predicting it earlier – say, before we willingly handed all our personal information to our smart devices.

For the moment, it seems our more stringent ethical controls should delay the onset of any such social credit system in the West. But it seems fairly clear that were Apple, Google, Facebook or Amazon to decide they wanted to mine our personal information, we would be completely powerless to stop them. What do you think?