Machine learning algorithms are increasingly used to support decision-making in the exercise of public authority. Here, we argue that an important consideration has been overlooked in previous discussions: whether the use of ML undermines the democratic legitimacy of public institutions. From the perspective of democratic legitimacy, it is not enough that ML contributes to efficiency and accuracy in the exercise of public authority, which has so far been the focus in the scholarly literature engaging with these developments. According to one (...) influential theory, exercises of administrative and judicial authority are democratically legitimate if and only if administrative and judicial decisions serve the ends of the democratic law maker, are based on reasons that align with these ends and are accessible to the public. These requirements are not satisfied by decisions determined through ML since such decisions are determined by statistical operations that are opaque in several respects. However, not all ML-based decision support systems pose the same risk, and we argue that a considered judgment on the democratic legitimacy of ML in exercises of public authority need take the complexity of the issue into account. This paper outlines considerations that help guide the assessment of whether a ML undermines democratic legitimacy when used to support public decisions. We argue that two main considerations are pertinent to such normative assessment. The first is the extent to which ML is practiced as intended and the extent to which it replaces decisions that were previously accessible and based on reasons. The second is that uses of ML in exercises of public authority should be embedded in an institutional infrastructure that secures reason giving and accessibility. (shrink)

Machine learning algorithms pervade contemporary society. They are integral to social institutions, inform processes of governance, and animate the mundane technologies of daily life. Consistently, the outcomes of machine learning reflect, reproduce, and amplify structural inequalities. The field of fair machine learning has emerged in response, developing mathematical techniques that increase fairness based on anti-classification, classification parity, and calibration standards. In practice, these computational correctives invariably fall short, operating from an algorithmic idealism that does not, and cannot, address systemic, Intersectional (...) stratifications. Taking present fair machine learning methods as our point of departure, we suggest instead the notion and practice of algorithmic reparation. Rooted in theories of Intersectionality, reparative algorithms name, unmask, and undo allocative and representational harms as they materialize in sociotechnical form. We propose algorithmic reparation as a foundation for building, evaluating, adjusting, and when necessary, omitting and eradicating machine learning systems. (shrink)

In the age of Big Data, companies and governments are increasingly using algorithms to inform hiring decisions, employee management, policing, credit scoring, insurance pricing, and many more aspects of our lives. Artificial intelligence systems can help us make evidence-driven, efficient decisions, but can also confront us with unjustified, discriminatory decisions wrongly assumed to be accurate because they are made automatically and quantitatively. It is becoming evident that these technological developments are consequential to people’s fundamental human rights. Despite increasing attention to (...) these urgent challenges in recent years, technical solutions to these complex socio-ethical problems are often developed without empirical study of societal context and the critical input of societal stakeholders who are impacted by the technology. On the other hand, calls for more ethically and socially aware AI often fail to provide answers for how to proceed beyond stressing the importance of transparency, explainability, and fairness. Bridging these socio-technical gaps and the deep divide between abstract value language and design requirements is essential to facilitate nuanced, context-dependent design choices that will support moral and social values. In this paper, we bridge this divide through the framework of Design for Values, drawing on methodologies of Value Sensitive Design and Participatory Design to present a roadmap for proactively engaging societal stakeholders to translate fundamental human rights into context-dependent design requirements through a structured, inclusive, and transparent process. (shrink)

Normative debates on migration control, while characterised by profound disagreement, do appear to agree that the state has at least a prima facie right to prevent the entry of security threats. While concern is sometimes raised that this ‘security exception’ can be abused, there has been little focus by normative theorists on concrete practices of security, and how we can determine what a ‘principled’ use of the security exception would be. I argue that even if states have a right to (...) prevent migrants from entering on grounds of security, the practices required to implement this right will likely compromise core liberal democratic values in ways that have not been sufficiently appreciated. I draw on literature from Science and Technology Studies and Critical Security Studies to examine the proliferation of digital border control technologies which are increasingly dominating state security practices, and explain the challenges these technologies pose to non-discrimination, liberty and equality. I also highlight the difficulties of taking these principles as presumptive limitations on the operation of the security exception. A more sustained conversation among ethical theorists is needed to address the profound ethical challenges that the ever-increasing use of these technologies poses, particularly in pursuit of security. (shrink)

Novel technological devices, applications, and algorithms can provide us with a vast amount of personal information about ourselves. Given that we have ethical and practical reasons to pursue self-knowledge, should we use technology to increase our self-knowledge? And which ethical issues arise from the pursuit of technologically sourced self-knowledge? In this paper, I explore these questions in relation to bioinformation technologies (health and activity trackers, DTC genetic testing, and DTC neurotechnologies) and algorithmic profiling used for recommender systems, targeted advertising, and (...) technologically supported decision-making. First, I distinguish between impersonal, critical, and relational self-knowledge. Relational self-knowledge is a so far neglected dimension of self-knowledge which is introduced in this paper. Next, I investigate the contribution of these technologies to the three types of self-knowledge and uncover the connected ethical concerns. Technology can provide a lot of impersonal self-knowledge, but we should focus on the quality of the information which tends to be particularly insufficient for marginalized groups. In terms of critical self-knowledge, the nature of technologically sourced personal information typically impedes critical engagement. The value of relational self-knowledge speaks in favour of transparency of information technology, notably for algorithms that are involved in decision-making about individuals. Moreover, bioinformation technologies and digital profiling shape the concepts and norms that define us. We should ensure they not only serve commercial interests but our identity and self-knowledge interests. (shrink)

People's sense of being subject to digital dataveillance can cause them to restrict their digital communication behavior. Such a chilling effect is essentially a form of self-censorship in everyday digital media use with the attendant risks of undermining individual autonomy and well-being. This article combines the existing theoretical and limited empirical work on surveillance and chilling effects across fields with an analysis of novel data toward a research agenda. The institutional practice of dataveillance—the automated, continuous, and unspecific collection, retention, and (...) analysis of digital traces—affects individual behavior. A mechanism-based causal model based on the theory of planned behavior is proposed for the micro level: An individual's increased sense of dataveillance causes their subjective probability assigned to negative outcomes of digital communication behavior to increase and attitudes toward this communication to become less favorable, ultimately decreasing the intention to engage in it. In aggregate and triggered through successive salience shocks such as data scandals, dataveillance is accordingly hypothesized to lower the baseline of free digital communication in a society through the chilling effects mechanism. From the developed theoretical model, a set of methodological consequences and questions for future studies are derived. (shrink)

This article advocates for a hermeneutic model for children-AI interactions in which the desirable purpose of children’s interaction with artificial intelligence systems is children's growth. The article perceives AI systems with machine-learning components as having a recursive element when interacting with children. They can learn from an encounter with children and incorporate data from interaction, not only from prior programming. Given the purpose of growth and this recursive element of AI, the article argues for distinguishing the interpretation of bias within (...) the artificial intelligence ethics and responsible AI discourse. Interpreting bias as a preference and distinguishing between positive and negative bias is needed as this would serve children's healthy psychological and moral development. The human-centric AI discourse advocates for an alignment of capacities of humans and capabilities of machines by a focus both on the purpose of humans and on the purpose of machines for humans. The emphasis on mitigating negative biases through data protection, AI law, and certain value-sensitive design frameworks demonstrates that the purpose of the machine for humans is prioritized over the purpose of humans. These top–down frameworks often narrow down the purpose of machines to do-no-harm and they miss accounting for the bottom-up views and developmental needs of children. Therefore, applying a growth model for children-AI interactions that incorporates learning from negative AI-mediated biases and amplifying positive ones would positively benefit children’s development and children-centric AI innovation. Consequently, the article explores: What challenges arise from mitigating negative biases and amplifying positive biases in children-AI interactions and how can a growth model address these? To answer this, the article recommends applying a growth model in open AI co-creational spaces with and for children. In such spaces human–machine and human–human value alignment methods can be collectively applied in such a manner that children can become sensitized toward the effects of AI-mediated negative biases on themselves and others; enable children to appropriate and imbue top-down values of diversity, and non-discrimination with their meanings; enforce children’s right to identity and non-discrimination; guide children in developing an inclusive mindset; inform top-down normative AI frameworks by children’s bottom-up views; contribute to design criteria for children-centric AI. Applying such methods under a growth model in AI co-creational spaces with children could yield an inclusive co-evolution between responsible young humans in the loop and children-centric AI systems. (shrink)

Academic and public debates are increasingly concerned with the question whether and how algorithmic decision-making may reinforce social inequality. Most previous research on this topic originates from computer science. The social sciences, however, have huge potentials to contribute to research on social consequences of ADM. Based on a process model of ADM systems, we demonstrate how social sciences may advance the literature on the impacts of ADM on social inequality by uncovering and mitigating biases in training data, by understanding data (...) processing and analysis, as well as by studying social contexts of algorithms in practice. Furthermore, we show that fairness notions need to be evaluated with respect to specific outcomes of ADM systems and with respect to concrete social contexts. Social sciences may evaluate how individuals handle algorithmic decisions in practice and how single decisions aggregate to macro social outcomes. In this overview, we highlight how social sciences can apply their knowledge on social stratification and on substantive domains of ADM applications to advance the understanding of social impacts of ADM. (shrink)