In the UK this week, it was revealed that thousands of train passengers have had their faces scanned by Amazon hosted AI systems that analyse demographics and emotions – without consent or public discussion. According to documents obtained by the civil liberties group Big Brother Watch through a freedom of information request, Network Rail, a government-owned company, conducted a pilot scheme at major UK train stations in 2022. The scheme involved using AI-powered cameras to analyse passengers’ emotions, determining whether they appeared happy, sad, or angry.
Jake Hurfurt, Head of Research & Investigations at Big Brother Watch, said: “Network Rail had no right to deploy discredited emotion recognition technology against unwitting commuters at some of Britain’s biggest stations, and I have submitted a complaint to the Information Commissioner about this trial. It is alarming that as a public body it decided to roll out a large-scale trial of Amazon-made AI surveillance in several stations with no public awareness, especially when Network Rail mixed safety tech in with pseudoscientific tools and suggested the data could be given to advertisers”.
It will come as a surprise to many that there is no regulation or legal restriction on this kind of large-scale surveillance. In the UK, the use of facial recognition technology and emotion analysis in public spaces remains unregulated. While the Data Protection Act 2018 and the UK General Data Protection Regulation provide some general principles for the processing of personal data, there are no specific laws addressing the unique challenges posed by AI-powered surveillance systems.
The UK and EU have taken divergent paths regarding the regulation of AI biometric surveillance. The EU has passed the AI Act, which will heavily restrict the use of AI surveillance, banning unwarranted use and imposing limits on its use by law enforcement. In contrast, the UK government had plans to expand AI surveillance for policing purposes, despite concerns raised by the former Biometric Surveillance Camera Commissioner about its reliability and fairness. The UK’s Data Protection and Digital Information Bill, which would have made significant changes to data protection regulations and oversight of biometric data, was dropped due to the July general election and will now never see the light of day. The future of AI surveillance regulation in the UK will depend on the priorities of the next government.
The situation is further complicated by the fact that the UK’s data protection authority, the Information Commissioner’s Office (ICO), has limited powers to enforce regulations against companies based outside the country, as demonstrated by the overturned Clearview AI fine. This gap leaves UK citizens vulnerable to having their personal data collected and processed by AI systems without their knowledge or consent.
Meanwhile AI researchers this week raised the alarm around weaknesses in machine perception. Experiments have shown that making tiny tweaks to images can cause even highly capable AI systems to confidently perceive things that aren’t there. Worryingly, these ‘adversarial attacks’ are getting better at targeting specific AI behaviours – fooling them into seeing or not seeing whatever the attacker wants. As AI-driven surveillance and monitoring proliferates, the implications of these perceptual weaknesses are a concern.
Takeaways: The deployment of AI surveillance technologies, such as facial recognition and emotion analysis, requires much greater transparency from the companies and organisations implementing them. Citizens should demand comprehensive regulations that mandate strict security measures, built-in rights safeguards, and robust oversight to govern the use of AI surveillance in both the public and private sectors. Legislation must be technology-neutral and adaptable to close loopholes.