From demonstrations to the Dutch childcare benefits scandal, AI and algorithms are everywhere, but how are they being perceived?

This article was first published by Erasmus School of Law.

The use of AI in surveillance is becoming an increasingly relevant topic in the context of security and in the fight against crime. Think of the use of biometrics, anomaly detection, DNA banking, crowd control, facial recognition, violence detection, 24-hour surveillance and other forms of data-intensive surveillance. These phenomena raise crucial questions about their effects on the human experience. This symposium provided an opportunity to explore various facets of AI surveillance and its implications on public safety. This included not only the technological and legal aspects, but also the emotional, social and cultural aspects that come with the use of AI.

The use of AI: from protest policing to the childcare benefits scandal

Protest policing: a cat-and-mouse game 2.0 

Protest policing is playing an increasing role in EU debates, especially as AI is being used in enforcement and surveillance. This presentation answered questions such as: How do the police prepare for protests, and does this lead to new experiences for protesters and police officers? How does the integration of technologies and AI mediate the relationship between protesters and police? And does this AI-driven approach lead to different dynamics of public demonstrations and the use of social media? 

Continuously seeking the balance 

Majsa Storbeck, PhD researcher at Erasmus School of Law, states: "Protest enforcement seeks to find the right balance between public order and protection of fundamental rights, which includes the right to demonstrate. Adding new surveillance technologies such as AI adds new weight to the scales, and in doing so, a new balance must be found." "The deployment of technology increases the visibility and mobilisation of people, which can result in more demonstrations, demonstrations with larger crowds, and more diverse demonstrations. On the enforcement side, technology improves crowd management, proactive policing and efficiency", Storbeck said. 

But how does this affect the dynamic? 

Storbeck researched the interaction between police and XR protesters and described it as a cat-and-mouse game that has been evolved: "The mouse, in this context the protesters, can deploy technology to change the cat's field of view. Whereas the cat, in this context, the police, with the deployment of technology, has new claws to catch the mouse." However, this could have significant implications for the interaction and effectiveness of protest policing. Where it can have substantial benefits, there also appear to be adverse effects associated with deploying technology and AI. "Think of increased tension and reinforcement of mutual distrust. Moreover, technology blurs the boundaries of when a protest begins and ends. Thus, significant risks may be associated with police deployment of new technologies. Although these tools are primarily used for crowd management and are not necessarily intended to increase police power, they may inadvertently reinforce the perception of an all-powerful entity. I hope the results of my research will contribute to a wider public debate on the use of technology within protest policing, and its effects", Storbeck said.  

"That is why it is essential that, both in parliament and within legislation, more attention is paid to the use of AI and other technologies in demonstrations. Only by carefully considering the consequences and establishing clear rules and regulations can we ensure that the police can use these new technologies in a legally responsible manner. Practice must not take precedence over law", Storbeck stated.

Affected by the childcare benefits scandal: how does it affect a victim?

The impact is significant

The conversation quickly reveals the impact of the childcare benefits scandal. The moment Brown and Strangio were labelled as fraudsters had a big impact on their lives. "As soon as you get a message like that, you start doubting yourself and researching if it is possible that you really made a mistake. You don't assume that the government could be wrong", Strangio stated. But it does not stop at the shock of being labelled a fraud. Proving innocence takes a long time and causes a lot of stress which has a major impact on the lives of those affected and on the lives of their families. Stijns adds: "People lost their homes, relationships were destroyed, and for many, it had a major impact on their health. In some cases, the problems became so severe that children were even removed from their homes." "You are labelled and treated as a fraud and have no money to fight for your rights. Your life is literally taken away from you. I was blacklisted, which means many things become nearly impossible", Brown said. Strangio's story also painfully reveals the long-term impact: "I was always very entrepreneurial and had two successful businesses before the childcare benefits scandal. Eventually, I had to give up my business and work in a call centre to make a living. It really took away my drive to be an entrepreneur. Because why build a business if everything can be taken away from you like that?" 
 
Misrepresentation of those affected 

Konaté, who wrote her thesis on the childcare benefits scandal, stressed that there is often a completely wrong perception of those who were affected. "In the Netherlands, we have a strong norm of an ideal citizen. Who is hardworking, has their affairs in order and is independent from government reimbursements. The vast majority of those affected by the childcare benefits scandal are definitely not fraudsters but also do not appear to fit this norm completely. Seventy per cent of the group affected appears to consist of people with a migration background, and fifty per cent of them are single-person households, mostly mothers, and socioeconomic status also plays a role in this. And that while these are actually hard-working parents, as you will not receive childcare benefit if you are not working."

How can we do better in future? 

"In an ideal world, free childcare would obviously be the most beautiful solution. But I understand this is impossible. It is important that the system of mistrust disappears. We need to re-evaluate this policy and make room for dissenting voices", Konaté said. In addition, Strangio stresses the importance of working on building an algorithm that is free of biases. 
 
"The current proposal by the outgoing cabinet to channel childcare subsidies directly through the childcare organisations can ensure that this situation is not repeated, but only if parents are then also completely removed from the process", Brown states. "We also hope that the world will be a better place and that we can discuss how to resolve and prevent this from happening again. That is why we have set up citizen panels in collaboration with the State Commission on the Rule of Law to discuss what could be improved in our constitutional state. We will present these recommendations to the Lower House in July", Brown said. Konaté finished: "It is essential to keep thinking about the human experience on the other side of a decision, technology or policy document." 

A new perspective 

According to Schuilenburg, the AI story is currently mainly focused on the technical side of it, about the unprecedented possibilities AI offers. Or on the question of what should be understood as 'intelligence' and how 'intelligent' AI tools actually are. "Those are important topics", Schuilenburg argues. "But we should not forget that AI is also always a social practice. Therefore, the experiences of people interacting with AI, from immigrants and police officers to platform workers and protesters of Extinction Rebellion, should be included in talking and thinking about AI." 

If this does not happen, Schuilenburg argues, individuals will be reduced to abstract profiles and thus become invisible and disappear from view. Therefore, putting the AI experiences in the centre of the discussion answers a societal need to give a voice to all those who do not have it in the field of AI or have it insufficiently think especially of 'silenced voices' of weak, vulnerable or marginalised groups. 

"The conference has shown that there is a great need to conduct more research on the AI experiences of citizens and professionals in order to better understand the perspectives and meanings individuals give to AI and algorithms and include them in the design and application of AI tools. We need to make AI 'public'”, Schuilenburg states.