In today’s highly dynamic societies, moral norms and values are subject to change. Moral change is partly driven by technological developments. For instance, the introduction of robots in elderly care practices requires caregivers to share moral responsibility with a robot (see van Wynsberghe 2013 ). Since we do not know what elements of morality will change and how they will change (see van der Burg 2003 ), moral education should aim at fostering what has been called “moral resilience” (Swierstra 2013 (...) ). We seek to fill two gaps in the existing literature: (i) research on moral education has not paid enough attention to the development of moral resilience; (ii) the very limited literature on moral resilience does not conceptualise moral resilience in relation to new technological developments. We argue that philosophical accounts of moral education need to do justice to the importance of moral resilience, and that a specific form of moral resilience should be conceptualised as “technomoral resilience” to underline the added value of cultivating moral resilience in relation to technomoral change. We illustrate the role of technomoral resilience in practice by looking at the context of elderly care. To make the first step towards an account of how technomoral resilience can be fostered in moral education, we propose that moral education shall focus on a triangle of capacities: (1) moral imagination, (2) a capacity for critical reflection, and (3) a capacity for maintaining one’s moral agency in the face of disturbances. (shrink)

To be successful, policy must anticipate a broad range of constituents. Yet, all too often, technology policy is written with primarily mainstream populations in mind. In this article, drawing on Value Sensitive Design and discount evaluation methods, we introduce a new method—Diverse Voices—for strengthening pre-publication technology policy documents from the perspective of underrepresented groups. Cost effective and high impact, the Diverse Voices method intervenes by soliciting input from “experiential” expert panels. We first describe the method. Then we report on two (...) case studies demonstrating its use: one with a white paper on augmented reality technology with expert panels on people with disabilities, people who were formerly or currently incarcerated, and women; and the other with a strategy document on automated driving vehicle technologies with expert panels on youth, non-car drivers, and extremely low-income people. In both case studies, panels identified significant shortcomings in the pre-publication documents which, if addressed, would mitigate some of the disparate impact of the proposed policy recommendations on these particular stakeholder groups. Our discussion includes reflection on the method, evidence for its success, its limitations, and future directions. (shrink)

Responsible innovation is playing a progressively significant role in the developed countries of Europe and North America, but is not as often used as a framework for analysis in developing countries. This article describes how what we term latent or implicit RI was introduced to a large-scale technological project in China: the expansion of the deep-sea port in Dalian. This analysis first reviews the historical development of the port and summarizes the Chinese context since the Reform and Opening that began (...) in the late 1970s. It provides a brief case study of a particular emergency containment pool construction project undertaken by Dalian Port, uses stakeholder analysis to evaluate the project, identifies key values influencing the work, and considers the extent to which it meets criteria set out in a specific four-dimensional RI methodological framework. We argue that although it does not use RI rhetoric, the PDA in effect practices latent RI which is also in fact becoming manifest. In conclusion we also consider some of the weaknesses of our argument and some ambiguities of RI. (shrink)

Franke Philosophy & Technology, 35(4), 1-7, (2022) offers an interesting claim that algorithmic transparency as manipulation does not necessarily follow that it is good or bad. Different people can have good reasons to adopt different evaluative attitudes towards this manipulation. Despite agreeing with some of his observations, this short reply will examine three crucial misconceptions in his arguments. In doing so, it defends why we are morally obliged to care about the manipulative potential of algorithmic transparency. It suggests that we (...) as society have a moral duty to incorporate the value of transparency into algorithmic systems while keeping algorithmic transparency itself sensitive to power relations. (shrink)

Rapid advancements in machine learning techniques allow mass surveillance to be applied on larger scales and utilize more and more personal data. These developments demand reconsideration of the privacy-security dilemma, which describes the tradeoffs between national security interests and individual privacy concerns. By investigating mass surveillance techniques that use bulk data collection and machine learning algorithms, we show why these methods are unlikely to pinpoint terrorists in order to prevent attacks. The diverse characteristics of terrorist attacks—especially when considering lone-wolf terrorism—lead (...) to irregular and isolated footprints. The irregularity of data affects the accuracy of machine learning algorithms and the mass surveillance that depends on them which can be explained by three kinds of known problems encountered in machine learning theory: class imbalance, the curse of dimensionality, and spurious correlations. Proponents of mass surveillance often invoke the distinction between collecting data and metadata, in which the latter is understood as a lesser breach of privacy. Their arguments commonly overlook the ambiguity in the definitions of data and metadata and ignore the ability of machine learning techniques to infer the former from the latter. Given the sparsity of datasets used for machine learning in counterterrorism and the privacy risks attendant with bulk data collection, policymakers and other relevant stakeholders should critically re-evaluate the likelihood of success of the algorithms and the collection of data on which they depend. (shrink)

The notion of responsible innovation suggests that innovators carry additional responsibilities beyond those commonly suggested. In this paper, we will discuss the meaning of these novel responsibilities focusing on two philosophical problems of attributing such responsibilities to innovators. The first is the allocation of responsibilities to innovators. Innovation is a process that involves a multiplicity of agents and unpredictable, far-reaching causal chains from innovation to social impacts, which creates great uncertainty. A second problem is constituted by possible trade-offs between different (...) kinds of responsibility. It is evident that attributing backward-looking responsibility for product failures diminishes the willingness to learn about such defects and to take forward-looking responsibility. We will argue that these problems can be overcome by elaborating what it is exactly that innovators are responsible for. In this manner, we will distinguish more clearly between holding responsible and taking responsibility. This opens a space for ‘supererogatory’ responsibilities. Second, we will argue that both innovation processes and outcomes can be objects of innovators’ responsibility. Third, we will analyze different kinds of responsibility and show that the functions of their attribution are not necessarily contradictory. Based on this conceptual refinement, we will argue that accountability, responsibility-as-virtue and the willingness to take responsibility are crucial for responsible innovation. (shrink)

In the past decades, computers have become more and more involved in society by the rise of ubiquitous systems, increasing the number of interactions between humans and IT systems. At the same time, the technology itself is getting more complex, enabling devices to act in a way that previously only humans could, based on developments in the fields of both robotics and artificial intelligence. This results in a situation in which many autonomous, intelligent and context-aware systems are involved in decisions (...) that affect their environment. These relations between people, machines, and decisions can take many different forms, but thus far, a systematic account of machine-assisted moral decisions is lacking. This paper investigates the concept of machine-assisted moral decisions from the perspective of technological mediation. It is argued that modern machines do not only have morality in the sense of mediating the actions of humans, but that, by making their own decisions within their relations with humans, mediate morality itself. A classification is proposed to differentiate between four different types of moral relations. The moral aspects within the decisions these systems make are combined into three dimensions that describe the distinct characteristics of different types of moral mediation by machines. Based on this classification, specific guidelines for moral behavior can be provided for these systems. (shrink)

Proactively including the ethical and societal issues of new technologies could have a positive effect on their acceptance. These issues could be captured in terms of values. In the literature, the values stakeholders deem important for the development of technology have often been identified. However, the relative ranking of these values in relation to each other have not been studied often. The best worst method is proposed as a possible method to determine the weights of values, hence it is used (...) in an evaluative fashion. The applicability of the method is tested by applying it to the case of smart meters, one of the main components of the smart grid. The importance of values is examined for three dimensions of acceptance namely sociopolitical, market, and household acceptance. (shrink)

Design for Values (DfV) philosophies are a series of design approaches that aim to incorporate human values into the early phases of technological design to direct innovation into beneficial outcomes. The difficulty and necessity of directing advantageous futures for transformative technologies through the application and adoption of value-based design approaches are apparent. However, questions of whose values to design are of critical importance. DfV philosophies typically aim to enrol the stakeholders who may be affected by the emergence of such a (...) technology. However, regardless of which design approach is adopted, all enrolled stakeholders are human ones who propose human values. Contemporary scholarship on metahumanisms, particularly those on posthumanism, have decentred the human from its traditionally privileged position among other forms of life. Arguments that the humanist position is not (and has never been) tenable are persuasive. As such, scholarship has begun to provide a more encompassing ontology for the investigation of nonhuman values. Given the potentially transformative nature of future technologies as relates to the earth and its many assemblages, it is clear that the value investigations of these design approaches fail to account for all relevant stakeholders (i.e., nonhuman animals). This paper has two primary objectives: (1) to argue for the cogency of a posthuman ethics in the design of technologies; and (2) to describe how existing DfV approaches can begin to envision principled and methodological ways of incorporating non-human values into design. To do this, the paper provides a rudimentary outline of what constitutes DfV approaches. It then takes up a unique design approach called Value Sensitive Design (VSD) as an illustrative example. Out of all the other DfV frameworks, VSD most clearly illustrates a principled approach to the integration of values in design. (shrink)

Safe-by-Design (SBD) frameworks for the development of emerging technologies have become an ever more popular means by which scholars argue that transformative emerging technologies can safely incorporate human values. One such popular SBD methodology is called Value Sensitive Design (VSD). A central tenet of this design methodology is to investigate stakeholder values and design those values into technologies during early stage research and development (R&D). To accomplish this, the VSD framework mandates that designers consult the philosophical and ethical literature to (...) best determine how to weigh moral trade-offs. However, the VSD framework also concedes the universalism of moral values, particularly the values of freedom, autonomy, equality trust and privacy justice. This paper argues that the VSD methodology, particularly applied to nano-bio-info-cogno (NBIC) technologies, has an insufficient grounding for the determination of moral values. As such, an exploration of the value-investigations of VSD are deconstructed to illustrate both its strengths and weaknesses. This paper also provides possible modalities for the strengthening of the VSD methodology, particularly through the application of moral imagination and how moral imagination exceed the boundaries of moral intuitions in the development of novel technologies. (shrink)

Healthcare is becoming increasingly automated with the development and deployment of care robots. There are many benefits to care robots but they also pose many challenging ethical issues. This paper takes care robots for the elderly as the subject of analysis, building on previous literature in the domain of the ethics and design of care robots. Using the value sensitive design approach to technology design, this paper extends its application to care robots by integrating the values of care, values that (...) are specific to AI, and higher-scale values such as the United Nations Sustainable Development Goals. The ethical issues specific to care robots for the elderly are discussed at length alongside examples of specific design requirements that work to ameliorate these ethical concerns. (shrink)

The adverse effects of artificial nighttime lighting, known as light pollution, are emerging as an important environmental issue. To address these effects, current scientific research focuses mainly on identifying what is bad or undesirable about certain types and uses of lighting at night. This paper adopts a value-sensitive approach, focusing instead on what is good about darkness at night. In doing so, it offers a first comprehensive analysis of the environmental value of darkness at night from within applied ethics. A (...) design for values orientation is utilized to conceptualize, define, and categorize the ways in which value is derived from darkness. Nine values are identified and categorized via their type of good, temporal outlook, and spatial characteristics. Furthermore, these nine values are translated into prima facie moral obligations that should be incorporated into future design choices, policy-making, and innovations to nighttime lighting. Thus, the value of darkness is analyzed with the practical goal of informing future decision-making about urban nighttime lighting. (shrink)

The concept of mutual responsiveness is currently based on little empirical data in the literature of Responsible Research and Innovation. This paper explores RRI’s idea of mutual responsiveness in the light of recent RRI case studies on private sector research and development. In RRI, responsible innovation is understood as a joint endeavour of innovators and societal stakeholders, who become mutually responsive to each other in defining the ‘right impacts’ of the innovation in society, and in steering the innovation towards realising (...) those impacts. Yet, the case studies identified several reasons for why the idea of mutual responsiveness does not always appear feasible or desirable in actual R&D situations. Inspired by the discrepancies between theory and practice, we suggest three further elaborations for the concept of responsiveness in RRI. Process-responsiveness is suggested for identifying situations that require stakeholder involvement specifically during R&D. Product-responsiveness is suggested for mobilising the potential of innovation products to be adaptable according to diverse stakeholder needs. Presponsiveness is suggested as responsiveness towards stakeholders that are not reachable at a given time of R&D. Our aim is to contribute to a more tangible understanding of responsiveness in RRI, and suggest directions for further analysis in upcoming RRI case studies. (shrink)

The paper has two goals. The first is presenting the main results of the recent report Ethics of Connected and Automated Vehicles: recommendations on road safety, privacy, fairness, explainability and responsibility written by the Horizon 2020 European Commission Expert Group to advise on specific ethical issues raised by driverless mobility, of which the author of this paper has been member and rapporteur. The second is presenting some broader ethical and philosophical implications of these recommendations, and using these to contribute to (...) the establishment of Ethics of Transportation as an independent branch of applied ethics. The recent debate on the ethics of Connected and Automated Vehicles presents a paradox and an opportunity. The paradox is the presence of a flourishing debate on the ethics of one very specific transportation technology without ethics of transportation being in itself a well-established academic discipline. The opportunity is that now that a spotlight has been switched on the ethical dimensions of CAVs it may be easier to establish a broader debate on ethics of transportation. While the 20 recommendations of the EU report are grouped in three macro-areas: road safety, data ethics, and responsibility, in this paper they will be grouped according to eight philosophical themes: Responsible Innovation, road justice, road safety, freedom, human control, privacy, data fairness, responsibility. These are proposed as the first topics for a new ethics of transportation. (shrink)

The notion of “responsibility gap” with artificial intelligence (AI) was originally introduced in the philosophical debate to indicate the concern that “learning automata” may make more difficult or impossible to attribute moral culpability to persons for untoward events. Building on literature in moral and legal philosophy, and ethics of technology, the paper proposes a broader and more comprehensive analysis of the responsibility gap. The responsibility gap, it is argued, is not one problem but a set of at least four interconnected (...) problems – gaps in culpability, moral and public accountability, active responsibility—caused by different sources, some technical, other organisational, legal, ethical, and societal. Responsibility gaps may also happen with non-learning systems. The paper clarifies which aspect of AI may cause which gap in which form of responsibility, and why each of these gaps matter. It proposes a critical review of partial and non-satisfactory attempts to address the responsibility gap: those which present it as a new and intractable problem (“fatalism”), those which dismiss it as a false problem (“deflationism”), and those which reduce it to only one of its dimensions or sources and/or present it as a problem that can be solved by simply introducing new technical and/or legal tools (“solutionism”). The paper also outlines a more comprehensive approach to address the responsibility gaps with AI in their entirety, based on the idea of designing socio-technical systems for “meaningful human control", that is systems aligned with the relevant human reasons and capacities. (shrink)

The present debate over the creation and potential deployment of lethal autonomous weapons, or ‘killer robots’, is garnering more and more attention. Much of the argument revolves around whether such machines would be able to uphold the principle of noncombatant immunity. However, much of the present debate fails to take into consideration the practical realties of contemporary armed conflict, particularly generating military objectives and the adherence to a targeting process. This paper argues that we must look to the targeting process (...) if we are to gain a fuller picture of the consequences of creating or fielding lethal autonomous robots. This paper argues that once we look to how militaries actually create military objectives, and thus identify potential targets, we face an additional problem: the Strategic Robot Problem. The ability to create targeting lists using military doctrine and targeting processes is inherently strategic, and handing this capability over to a machine undermines existing comman.. (shrink)

In this contribution, we set out a framework for ethical research and innovation. Our framework draws upon recent scholarly work recommending the introduction of new models at the intersection of ethics, strategy, and science and technology studies to inform and explicate how the decisions of researchers can be considered ethical. Ethical research and innovation is construed in our framework as a dynamic process emerging from decisions of multiple stakeholders in innovation ecosystems prior to, during and after the execution of a (...) research and innovation project. The framework can be used by different types of research organizations to implement governance models of ethical research and innovation. (shrink)

Following societal and policy pressures for responsible innovation, innovators are more and more expected to consider the broader socio-ethical context of their work, and more importantly, to integrate such considerations into their daily practices. This may require the involvement of ‘outsiders’ in innovation trajectories, including e.g. societal and governmental actors. However, methods on how to functionally organize such integration in light of responsible innovation have only recently started to emerge. We present an approach to do just that, in which we (...) first develop value profiles of the involved actors, and second, design a workshop setting that allows innovators to develop design requirements in collaboration with representatives of parties that are not usually involved in such innovation design practices. Using a case study in automated vehicle development, we positively demonstrate the possibility and utility of our approach. We stress that in this study we wish to demonstrate the functionality of our developed method, and did not search for scientifically valid outcomes regarding this technical field. (shrink)

What does it mean to be irresponsible in developing or using a technology? There are two fundamentally different answers to this question and they each generate research strands that differ in scope, style and applicability. To capture this difference, I make use of two mythical creatures of Jewish origin that have been employed in the past to represent relationships between man and man-made entities: the Golem (Collins and Pinch, 2002, 2005 ) and the Leviathan (Hobbes, 1994 ). The Golem is (...) the traditional image of technology as a creature that can be helpful but needs to be controlled. Irresponsibility in this perspective is the failure to exercise control. The Leviathan is the image of technology as a difficult compromise between fundamental values. Irresponsibility is in this perspective is allowing some values to systematically dominate others. Having worked out the basics of these images, I show that each comes with its specific methodological challenges: where the Golem gives rise to the Collingridge Dilemma of control, the Leviathan gives rise to Münchhausen’s trilemma of justification. Since the Golem image is predominant in scholarship on irresponsibility, I conclude with an appeal for a more equal distribution of efforts in conceptualizing technologies as Golems and as Leviathans. (shrink)

Three philosophical perspectives on the relation between technology and society are distinguished and discussed: 1) technology as an autonomous force that determines society; 2) technology as a human construct that can be shaped by human values, and 3) a co-evolutionary perspective on technology and society where neither of them determines the other. The historical evolution of the three perspectives is discussed and it is argued that all three are still present in current debates about technological change and how it may (...) affect society. This is illustrated for the case of Artificial Intelligence (AI). It is argued that each of the three perspectives contributes to the debate of AI but that the third has the strongest potential to uncover blind spots in the current debate. (shrink)

In our high-tech society, the design process involves profound questions about the effects of the resulting goods, and the responsibilities of designers. In the philosophy of technology, effects of “things” on user experience and behaviour have been discussed in terms of the concept of technological mediation. Meanwhile, what we create has moved more and more towards services (processes) rather than products (things), in particular in the context of information services. The question is raised to what extent the concept of technological (...) mediation is adequate to understand effects and responsibilities in information services as well. Therefore, this paper discusses differences between product aspects and service aspects of our creations, and evaluates the applicability of the concept of technological mediation to information services. Specific features of a notion of technological mediation for information services are highlighted, in particular with respect to the different relation between production and consumption. Finally, the paper focuses on the ethical consequences of service impact, and recommendations for service providers, especially in terms of the possibilities for second-order mediation by inviting users to change service properties. (shrink)

Biobased production has been promoted as a sustainable alternative to fossil resources. However, controversies over its impact on sustainability highlight societal concerns, value tensions and uncertainties that have not been taken into account during its development. In this work, the consideration of stakeholders’ values in a biorefinery design project is investigated. Value sensitive design is a promising approach to the design of technologies with consideration of stakeholders’ values, however, it is not directly applicable for complex systems like biorefineries. Therefore, some (...) elements of VSD, such as the identification of relevant values and their connection to a technology’s features, are brought into biorefinery design practice. Midstream modulation, an approach to promoting the consideration of societal aspects during research and development activities, is applied to promote reflection and value considerations during the design decision making. As result, it is shown that MM interventions during the design process led to new design alternatives in support of stakeholders' values, and allowed to recognize and respond to emerging value tensions within the scope of the project. In this way, the present work shows a novel approach for the technical investigation of VSD, especially for biorefineries. Also, based on this work it is argued that not only reflection, but also flexibility and openness are important for the application of VSD in the context of biorefinery design. (shrink)

Robots are leaving factories and entering urban spaces. In this paper, I will explore how we can integrate robots of various types into the urban landscape. I will distinguish between two perspectives: the responsible design and use of urban robots and robots as part of responsible urban innovations. The first viewpoint considers issues arising from the use of a robot in an urban environment. To develop a substantive understanding of Responsible Urban Robotics, we need to focus on normative implications of (...) city life as the context in which in robots are being used. I will refer to the desirable qualities of city life as “cityness” and will argue that we should design for cityness. The second approach asks how robots might be used to address challenges specific to cities. From the perspective of RRI, this may require participatory approaches in which the needs of the stakeholders are addressed. But we may also find inspiration in the work undertaken in architecture on expanding the concept and field to ensure that architects not only provide services to the lucky few but also create useful and beautiful spaces for the many. A dialogue with architects, urban designers, and urban planners may also be needed to successfully address the spatial issues raised by the presence of robots in the city. (shrink)

The convergence of computing, sensing, and communication technology will soon permit large-scale deployment of self-driving vehicles. This will in turn permit a radical transformation of traffic control technology. This paper makes a case for the importance of addressing questions of social justice in this transformation, and sketches a preliminary framework for doing so. We explain how new forms of traffic control technology have potential implications for several dimensions of social justice, including safety, sustainability, privacy, efficiency, and equal access. Our central (...) focus is on efficiency and equal access as desiderata for traffic control design. We explain the limitations of conventional traffic control in meeting these desiderata, and sketch a preliminary vision for a next-generation traffic control tailored to address better the demands of social justice. One component of this vision is cooperative, hierarchically distributed self-organization among vehicles. Another component of this vision is a priority system enabling selection of priority levels by the user for each vehicle trip in the network, based on the supporting structure of non-monetary credits. (shrink)

This paper critically assesses the possibility of moral enhancement with ambient intelligence technologies and artificial intelligence presented in Savulescu and Maslen (2015). The main problem with their proposal is that it is not robust enough to play a normative role in users’ behavior. A more promising approach, and the one presented in the paper, relies on an artifi-cial moral reasoning engine, which is designed to present its users with moral arguments grounded in first-order normative theories, such as Kantianism or utilitarianism, (...) that reason-responsive people can be persuaded by. This proposal can play a normative role and it is also a more promising avenue towards moral enhancement. It is more promising because such a system can be designed to take advantage of the sometimes undue trust that people put in automated technologies. We could therefore expect a well-designed moral reasoner system to be able to persuade people that may not be persuaded by similar arguments from other people. So, all things considered, there is hope in artificial intelli-gence for moral enhancement, but not in artificial intelligence that relies solely on ambient intelligence technologies. (shrink)

Changing values may give rise to intergenerational conflicts, like in the ongoing climate change and energy transition debate. This essay focuses on the interpretative question of how this value change can best be understood. To elucidate the interpretation of value change, two philosophical perspectives on value are introduced: Berlin’s value pluralism and Dworkin’s interpretivism. While both authors do not explicitly discuss value change, I argue that their perspectives can be used for interpreting value change in the case of climate change (...) and the energy transition. I claim that Berlin’s pluralistic account of value would understand the value change as an intergenerational conflict and therefore provide a too narrow and static ground for understanding ongoing value change. Instead, by exploring Dworkin’s standpoint in moral epistemology, this essay distills a more encompassing perspective on how values may relate, converge, overlap, and change, fulfilling their functions in the course of climate change and energy transition. This perspective is further detailed by taking inspiration from Shue’s work on the interpretation of equity in the climate change debate. I argue that the resulting perspective allows us to see value change as a gradual process rather than as a clash between generations and their values. (shrink)

The urban traffic environment is characterized by the presence of a highly differentiated pool of users, including vulnerable ones. This makes vehicle automation particularly difficult to implement, as a safe coordination among those users is hard to achieve in such an open scenario. Different strategies have been proposed to address these coordination issues, but all of them have been found to be costly for they negatively affect a range of human values (e.g. safety, democracy, accountability…). In this paper, we claim (...) that the negative value impacts entailed by each of these strategies can be interpreted as lack of what we call Meaningful Human Control over different parts of a sociotechnical system. We argue that Meaningful Human Control theory provides the conceptual tools to reduce those unwanted consequences, and show how “designing for meaningful human control” constitutes a valid strategy to address coordination issues. Furthermore, we showcase a possible application of this framework in a highly dynamic urban scenario, aiming to safeguard important values such as safety, democracy, individual autonomy, and accountability. Our meaningful human control framework offers a perspective on coordination issues that allows to keep human actors in control while minimizing the active, operational role of the drivers. This approach makes ultimately possible to promote a safe and responsible transition to full automation. (shrink)

Recent developments in theories for responsible innovation have focused on the importance of actively accounting for values in our technological designs. Leading among these theories is that of Value Sensitive Design which attempts to guide the design process on the basis of evaluative analysis. However, values often come into conflict and VSD has been criticized for not providing a proper method to resolve such inevitable conflicts. This paper examines three such methods and argues that although each has its merits, they (...) all fail to account for a common source of value conflicts known as value incommensurability. Drawing on literature from the field of axiology, this paper argues that by incorporating the evaluative relation of ‘parity’ each of these three methods, and the VSD framework in general, will be able to properly understand the relation which holds between conflicting design options stemming from the incommensurable of values and be able to guide designers in making rational decision in the face of such conflicts. (shrink)

Responsible Research and Innovation (‘RRI’) is a cross-cutting priority for scientific research in the European Union and beyond. This paper considers whether the way such research is organised and delivered lends itself to the aims of RRI. We focus particularly on international consortia, which have emerged as a common model to organise large-scale, multi-disciplinary research in contemporary biomedical science. Typically, these consortia operate through fixed-term contracts, and employ governance frameworks consisting of reasonably standard, modular components such as management committees, advisory (...) boards, and data access committees, to co-ordinate the activities of partner institutions and align them with funding agency priorities. These have advantages for organisation and management of the research, but can actively inhibit researchers seeking to implement RRI activities. Conventional consortia governance structures pose specific problems for meaningful public and participant involvement, data sharing, transparency, and ‘legacy’ planning to deal with societal commitments that persist beyond the duration of the original project. In particular, the ‘upstream’ negotiation of contractual terms between funders and the institutions employing researchers can undermine the ability for those researchers to subsequently make decisions about data, or participant remuneration, or indeed what happens to consortia outputs after the project is finished, and can inhibit attempts to make project activities and goals responsive to input from ongoing dialogue with various stakeholders. Having explored these challenges, we make some recommendations for alternative consortia governance structures to better support RRI in future. (shrink)

This article presents the framework Capability Sensitive Design (CSD), which consists of merging the design methodology Value Sensitive Design (VSD) with Martha Nussbaum's capability theory. CSD aims to normatively assess technology design in general, and technology design for health and wellbeing in particular. Unique to CSD is its ability to account for human diversity and to counter (structural) injustices that manifest in technology design. The basic framework of CSD is demonstrated by applying it to the hypothetical design case of a (...) therapy chatbot for mental health. By applying CSD to a design case, the merits of this new framework over the standard VSD approach become apparent. Also, the application demonstrates what a technology design would look like when attention is paid to capabilities right from the start of the design process. (shrink)

This article presents a conceptual investigation into the value impacts and relations of algorithms in the domain of justice and security. As a conceptual investigation, it represents one step in a value sensitive design based methodology. Here, we explicate and analyse the expression of values of accuracy, privacy, fairness and equality, property and ownership, and accountability and transparency in this context. We find that values are sensitive to disvalue if algorithms are designed, implemented or deployed inappropriately or without sufficient consideration (...) for their value impacts, potentially resulting in problems including discrimination and constrained autonomy. Furthermore, we outline a framework of conceptual relations of values indicated by our analysis, and potential value tensions in their implementation and deployment with a view towards supporting future research, and supporting the value sensitive design of algorithms in justice and security. (shrink)

This article presents a systematic literature review documenting how technical investigations have been adapted in value sensitive design (VSD) studies from 1996 to 2023. We present a systematic review, including theoretical and applied studies that either discuss or conduct technical investigations in VSD. This systematic review contributes to the VSD community when seeking to further refine the methodological framework for carrying out technical investigations in VSD.

In this article, we report on eight grand challenges for value sensitive design, which were developed at a one-week workshop, Value Sensitive Design: Charting the Next Decade, Lorentz Center, Leiden, The Netherlands, November 14–18, 2016. A grand challenge is a substantial problem, opportunity, or question that motives sustained research and design activity. The eight grand challenges are: Accounting for Power, Evaluating Value Sensitive Design, Framing and Prioritizing Values, Professional and Industry Appropriation, Tech policy, Values and Human Emotions, Value Sensitive Design (...) and Intelligent Algorithms, and Value Tensions. Each grand challenge consists of a discussion of its importance and a set of tractable key questions. (shrink)

Moral bioenhancement, nudge-designed environments, and ambient persuasive technologies may help people behave more consistently with their deeply held moral convictions. Alternatively, they may aid people in overcoming cognitive and affective limitations that prevent them from appreciating a situation’s moral dimensions. Or they may simply make it easier for them to make the morally right choice by helping them to overcome sources of weakness of will. This paper makes two assumptions. First, technologies to improve people’s moral capacities are realizable. Second, such (...) technologies will actually help people get morality right and behave more consistently with whatever the ‘real’ right thing to do turns out to be. The paper then considers whether or not humanity loses anything valuable, particularly opportunities for moral progress, when being moral is made much easier by eliminating difficult moral deliberation and internal moral struggle. Ultimately, the worry that moral struggle has value as a catalyst for moral progress is rejected. Moral progress is understood here as the discovery and application of new values or sensitization to new sources of harm. (shrink)

The increase in the spread of conversational agents urgently requires to tackle the ethical issues linked to their design. In fact, developers frequently include in their products cues that trigger social biases in order to maximize the performance and the quality of human-machine interactions. The present paper discusses whether and to what extent it is ethically sound to intentionally trigger gender biases through the design of virtually embodied conversational agents. After outlining the complex dynamics involving social biases, social robots, and (...) design, we evaluate the ethics of integrating gender cues in conversational agents, analysing four different approaches to the problem. Finally, we suggest which approach in our opinion might have the best chances to reduce the negative effects of biases and discriminatory visions of gender dynamics. (shrink)

Following societal and policy pressures for responsible innovation, innovators are more and more expected to consider the broader socio-ethical context of their work, and more importantly, to integrate such considerations into their daily practices. This may require the involvement of ‘outsiders’ in innovation trajectories, including e.g. societal and governmental actors. However, methods on how to functionally organize such integration in light of responsible innovation have only recently started to emerge. We present an approach to do just that, in which we (...) first develop value profiles of the involved actors, and second, design a workshop setting that allows innovators to develop design requirements in collaboration with representatives of parties that are not usually involved in such innovation design practices. Using a case study in automated vehicle development, we positively demonstrate the possibility and utility of our approach. We stress that in this study we wish to demonstrate the functionality of our developed method, and did not search for scientifically valid outcomes regarding this technical field. (shrink)

Editors’ Overview: Moral Responsibility in Technology and Engineering Content Type Journal Article Category Original Paper Pages 1-11 DOI 10.1007/s11948-011-9285-z Authors Neelke Doorn, Department of Technology, Policy and Management, Delft University of Technology, P.O. Box 5015, 2600 GA Delft, The Netherlands Ibo van de Poel, Department of Technology, Policy and Management, Delft University of Technology, P.O. Box 5015, 2600 GA Delft, The Netherlands Journal Science and Engineering Ethics Online ISSN 1471-5546 Print ISSN 1353-3452 Journal Volume Volume 18 Journal Issue Volume 18, (...) Number 1. (shrink)

Fundamental questions in value sensitive design include whether and how high-tech products/artefacts could embody values and ethical ideals, and how plural and incommensurable values of ethical and social importance could be chosen rationally and objectively at a collective level. By using a humanitarian cargo drone study as a starting point, this paper tackles the challenges that VSD’s lack of commitment to a specific ethical theory generates in practical applications. Besides, it highlights how mainstream ethical approaches usually related to VSD are (...) incapable of solving main ethical dilemmas raised by technological design for well-being in democratic settings. Accordingly, it is argued that VSD’s ethical-democratic import would substantially be enhanced by the espousal of a procedural ethics stance and the deliberative approach to value and welfare entailed by Amartya Sen’s capability approach. Differently from rival ethical–political theories, its normative and meta-ethical foundations better handle human diversity, value-goal pluralism, conflicting vested interests as well as the epistemic-moral disagreements typical of contemporary complex democracies. Particularly, Sen’s capability approach procedural-deliberative tenets result in an “objective-impartial” choice procedure selecting a “hierarchy” of plural incommensurable values and rational goals thus, suitable to validate an applied science such as welfare-oriented technological design in concrete social environments. Conclusions suggest that refining VSD with a capability-based procedural approach to ethics fosters the concern for democracy and social justice while preserving vital scientific-technical standards. Major advantages are at an applied level to delivering ethically and socially justified, but yet highly functional technologies and high-tech products/artefacts. (shrink)

Value Sensitive Design (VSD) is a design methodology developed by Batya Friedman and Peter Kahn (2003) that brings in moral deliberations in an early stage of a design process. It assumes that neither technology itself is value neutral, nor shifts the value-ladennes to the sole usage of technology. This paper adds to emerging literature onVSD for autonomous weapons systems development and discusses extant literature on values in autonomous systems development in general and in autonomous weapons development in particular. I identify (...) opportunities, such as public debates, and threats, such as the classified nature of the design process, for VSD in autonomous weapons development. This paper contributes to academic debates about the non-neutrality of technology by pointing out that values have been and can be explicitly designed into autonomous systems. It is informative for policy makers and designers who are tasked with developing actual autonomous weapons or policies around such systems, as they learn about an established design methodology that is sensitive to societal concerns and industry needs and that can be applied to autonomous weapons systems. (shrink)

Virtue accounts of innovation ethics have recognized the virtue of creativity as an admirable trait in innovators. However, such accounts have not paid sufficient attention to the way creativity functions as a collective phenomenon. We propose a collective virtue account to supplement existing virtue accounts. We base our account on Kieran’s definition of creativity as a virtue and distinguish three components in it: creative output, mastery and intrinsic motivation. We argue that all of these components can meaningfully be attributed to (...) innovation groups. This means that we can also attribute the virtue of creativity to group agents involved in innovation. Recognizing creativity as a collective virtue in innovation is important because it allows for a more accurate evaluation of how successful innovation generally happens. The innovator who takes a collective virtue account of creativity seriously will give attention to the facilitation of an environment where the group can flourish collectively, rather than only nurturing the individual genius. (shrink)

The ethics of technology has primarily focused on what values are and how they can be embedded in technologies through design. In this context, some work has been done to show the efficacy of a number of design approaches. However, existing studies have not clearly pointed out the ways in which design-for-values approaches can be used by design team managers to properly organize and use technologies in practice. This chapter attempts to fill this gap by discussing the value sensitive design (...) (VSD) approach as a useful means of co-designing technologies as a toolkit for existing workflow management, in this case, Agile. It will be demonstrated that VSD shows promise as a way of democratically designing technologies as well as fostering democratic technology policy innovation. (shrink)

The ethics of emerging forms of artificial intelligence has become a prolific subject in both academic and public spheres. A great deal of these concerns flow from the need to ensure that these technologies do not cause harm—physical, emotional or otherwise—to the human agents with which they will interact. In the literature, this challenge has been met with the creation of artificial moral agents: embodied or virtual forms of artificial intelligence whose decision procedures are constrained by explicit normative principles, requiring (...) the implementation of what is commonly called artificial morality into these agents. To date, the types of reasoning structures and principles which inform artificial morality have been of two kinds: first, an ethically maximal vision of artificial morality which relies on the strict implementation of traditional moral theories such as Kantian deontology or Utilitarianism, and second, a more minimalist vision which applies stochastic AI techniques to large data sets of human moral preferences so as to illicit or intuit general principles and preferences for the design of artificial morality. Taken individually, each approach is unable to fully answer the challenge of producing inoffensive behavior in artificial moral agents, most especially since both forms are unable to strike a balance between the ideal set of constraints which morality imposes on one hand, and the types of constraints public acceptability imposes, on the other. We provide an alternative approach to the design of artificial morality, the Ethical Valence Theory, whose purpose is to accommodate this balance, and apply this approach to the case of autonomous vehicles. (shrink)

Since its inception, the trolley problem has sparked a rich debate both within and beyond moral philosophy. Often used as a primer for students to begin thinking about moral intuitions as well as how to distinguish between different forms of moral reasoning, the trolley problem is not without its uses in very practical, applied field like engineering. Often thought of as unrealistic by technically-oriented engineers, trolley cases in fact, help us to think about moral responsibility in a high tech world. (...) This chapter explores the usefulness of trolley-like thinking within the realm of responsible innovation and discusses how despite the inherent issues with trolley scenarios, they remain nonetheless an indispensable tool for helping us to explore ways to maximize our moral responsibility in innovation. (shrink)

The power of technology to transform religions, science, and political institutions has often been presented as nothing short of revolutionary. Does technology have a similarly transformative influence on societies’ morality? Scholars have not rigorously investigated the role of technology in moral revolutions, even though existing research on technomoral change suggests that this role may be considerable. In this paper, we explore what the role of technology in moral revolutions, understood as processes of radical group-level moral change, amounts to. We do (...) so by investigating four historical episodes of radical moral change in which technology plays a noteworthy role. Our case-studies illustrate the plurality of mechanisms involved in technomoral revolutions, but also suggest general patterns of technomoral change, such as technology’s capacity to stabilize and destabilize moral systems, and to make morally salient phenomena visible or invisible. We find several leads to expand and refine conceptual tools for analysing moral change, specifically by crystallizing the notions of ‘technomoral niche construction’ and ‘moral payoff mechanisms’. Coming to terms with the role of technology in radical moral change, we argue, enriches our understanding of moral revolutions, and alerts us to the depths of which technology can change our societies in wanted and unwanted ways. (shrink)

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention model for (...) autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles. (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS) as well as the call for a ban are primarily focused on the nebulous concept of fully autonomous AWS. More specifically, on AWS that are capable of target selection and engagement without human supervision or control. This thesis argues that such a conception of autonomy is divorced both from military planning and decision-making operations as well as the design requirements that govern AWS engineering and subsequently the tracking (...) and tracing of moral responsibility. To do this, this thesis marries two different levels of meaningful human control (MHC), termed levels of abstraction, to couple military operations with design ethics. In doing so, this thesis argues that the contentious notion of ‘full’ autonomy is not problematic under this two-tiered understanding of MHC. It proceeds to propose the value sensitive design (VSD) approach as a means for designing for MHC. (shrink)