I recently read a paper with a truly dystopian theme that reminded me of the bureaucratic interactions depicted in the new Alien: Earth series. Just as the show portrays emotionless bureaucrats delivering devastating news without hesitation, the paper reveals a similarly unsettling reality within public employment services. It focuses on the role of human oversight in the operation of algorithms within these services. Specifically, it investigates a paradox:
“Ministerial statistics indicated an extremely low level of the corrections of algorithmic advice, respectively 0.58% 6 months after introduction of ADM and 3.68% 2 years later. What seemed puzzling in the light of almost inexistent corrections was the fact that caseworkers described profiling as “unreliable” and “inhuman”, and criticised ADM for failing to consider individual circumstances[…] Survey results confirm that many caseworkers regularly doubted ADM accuracy: almost 38% of the respondents perceived automatic classification as erroneous in at least half of the profiling cases. Less than 3% always found ADM accurate.”
In short, bureaucrats distrust algorithms, yet largely follow them. This is a rather dystopian observation, perhaps even cyberpunk. The paper attempts to gain a better understanding of this paradox. Great work, Karolina Sztandar-Sztanderska!
Link to the paper (behind a paywall).
Image: Nano Banana
Originally published on LinkedIn