Systems, machines and organisation of forms developed in the Engineering and Manufacturing sectors frequently lay the basis for systems design philosophy at a general level. An analysis of technological change in these sectors reveals that the resultant deskilling is not limited to the shop floor and is now spreading to intellectual work. The impact of ‘machine based systems’ on designers is explored in some detail and suggests the need for alternatives which are based on ‘ (...) class='Hi'>human centred systems’. Some attempts to design sub-systems are described. (shrink)

It is held that the quality of the working environment afforded to an individual critically affects the health and well-being of that individual. This has consequences for both the quality of work which that individual can actually perform, and for the quality of the society in which that individual has a place. Conceptions of a fit working environment have led to the idea of a human-centred system, and this idea is applicable to the area of knowledge-based systems (KBS). (...) A system structure is discussed, which permits a human-centred approach to be adopted towards KBS design. It is held that the performance of such a system will generally exceed that of unaided KBS operation. (shrink)

The urban traffic environment is characterized by the presence of a highly differentiated pool of users, including vulnerable ones. This makes vehicle automation particularly difficult to implement, as a safe coordination among those users is hard to achieve in such an open scenario. Different strategies have been proposed to address these coordination issues, but all of them have been found to be costly for they negatively affect a range of human values (e.g. safety, democracy, accountability…). In this paper, we (...) claim that the negative value impacts entailed by each of these strategies can be interpreted as lack of what we call Meaningful Human Control over different parts of a sociotechnical system. We argue that Meaningful Human Control theory provides the conceptual tools to reduce those unwanted consequences, and show how “designing for meaningful human control” constitutes a valid strategy to address coordination issues. Furthermore, we showcase a possible application of this framework in a highly dynamic urban scenario, aiming to safeguard important values such as safety, democracy, individual autonomy, and accountability. Our meaningful human control framework offers a perspective on coordination issues that allows to keep human actors in control while minimizing the active, operational role of the drivers. This approach makes ultimately possible to promote a safe and responsible transition to full automation. (shrink)

If our choices make us who we are, then what does that mean when these choices are made in the human-machine interface? Developing a clear understanding of how human decision making is influenced by automated systems in the environment is critical because, as human-machine interfaces and assistive robotics become even more ubiquitous in everyday life, many daily decisions will be an emergent result of the interactions between the human and the machine – (...) not stemming solely from the human. For example, choices can be influenced by the relative locations and motor costs of the response options, as well as by the timing of the response prompts. In drift diffusion model simulations of response-prompt timing manipulations, we find that it is only relatively equibiased choices that will be successfully influenced by this kind of perturbation. However, with drift diffusion model simulations of motor cost manipulations, we find that even relatively biased choices can still show some influence of the perturbation. We report the results of a two-alternative forced-choice experiment with a computer mouse modified to have a subtle velocity bias in a pre-determined direction for each trial, inducing an increased motor cost to move the cursor away from the pre-designated target direction. With queries that have each been normed in advance to be equibiased in people’s preferences, the participant will often begin their mouse movement before their cognitive choice has been finalized, and the directional bias in the mouse velocity exerts a small but significant influence on their final choice. With queries that are not equibiased, a similar influence is observed. By exploring the synergies that are developed between humans and machines and tracking their temporal dynamics, this work aims to provide insight into our evolving decisions. (shrink)

Generative art theory involves any artistic practice where an artist uses external systems (rules, algorithms, organic processes) to complete an artwork. Commonly utilized in video game design, gen...

Although even very advanced artificial systems do not meet the demanding conditions which are required for humans to be a proper participant in a social interaction, we argue that not all human-machine interactions (HMIs) can appropriately be reduced to mere tool-use. By criticizing the far too demanding conditions of standard construals of intentional agency we suggest a minimal approach that ascribes minimal agency to some artificial systems resulting in the proposal of taking minimal joint actions as (...) a case of a social HMI. Analyzing such HMIs, we utilize Dennett’s stance epistemology, and argue that taking either an intentional stance or design stance can be misleading for several reasons, and instead propose to introduce a new stance that is able to capture social HMIs—the AI-stance. (shrink)

Social robots are designed to perform intelligent, emotional, and autonomous behavior in order to establish intimate relationships with humans, for instance, in the context of elderly care. However, the imitation of qualities usually assumed to be necessary for human reciprocal interaction may impact our understanding of social interaction. Against this background, we compare the technical operations based on which social robots imitate human-like behavior with the concepts of emotionality, intelligence, and autonomy as usually attached to humans. In doing (...) so, we aim at making a conceptual contribution to the intuition that the technical imitation of emotional, intelligent, and autonomous behavior differs qualitatively from human forms of emotionality, intelligence, and autonomy. We conclude that the increasing tendency of social robots to imitate human characteristics and the thus increased human tendency of anthropomorphizing technical systems put forth new forms of human–machine interactions (HMI) that call for a critical examination. Thus, the emerging forms of interaction may lead, for instance, to false, reinforced, and perplexing assumptions regarding reciprocity. Therefore, studies are needed that critically examine the potential consequences and psychological effects that may be associated with the use of social robots, especially with regard to vulnerable groups such as children or the elderly. (shrink)

Clinical ethics can be viewed as a practical discipline that provides a structured approach to assist healthcare practitioners in identifying, analysing and resolving ethical issues that arise in practice. Clinical ethics can therefore promote ethically sound clinical and organisational practices and decision-making, thereby contributing to health organisation and system quality improvement. In order to develop students’ decision-making skills, as well as prepare them for practice, we decided to introduce a clinical ethics strand within an undergraduate medical curriculum. We designed a (...) programme of clinical ethics activities for teaching and assessment purposes that involved using ethical frameworks to analyse hypothetical and real-life cases in uni- and inter- professional groups. In this paper, we draw on medical student feedback collected over 6 years to illustrate the appeal to students of learning clinical ethics. We also outline the range of benefits for students, healthcare organisations, and the field of clinical ethics arising from tomorrow’s doctors experiencing clinical ethics early in their training. We conclude by briefly reflecting on how including clinical ethics within tomorrow’s doctors curricular can secure and continue future engagement in clinical ethics support services in the UK, alongside the dangers of preparing students for organisational cultures that might not exist. We anticipate the findings presented in the paper will contribute to wider debates examining the impact of ethics teaching, and its ability to inform future doctors’ practice. (shrink)

Human decisions are increasingly supported by decision support systems (DSS). Humans are required to remain “on the loop,” by monitoring and approving/rejecting machine recommendations. However, use of DSS can lead to overreliance on machines, reducing human oversight. This paper proposes “reflection machines” (RM) to increase meaningful human control. An RM provides a medical expert not with suggestions for a decision, but with questions that stimulate reflection about decisions. It can refer to data points or suggest (...) counterarguments that are less compatible with the planned decision. RMs think against the proposed decision in order to increase human resistance against automation complacency. Building on preliminary research, this paper will (1) make a case for deriving a set of design requirements for RMs from EU regulations, (2) suggest a way how RMs could support decision-making, (3) describe the possibility of how a prototype of an RM could apply to the medical domain of chronic low back pain, and (4) highlight the importance of exploring an RM’s functionality and the experiences of users working with it. (shrink)

As a first step in involving user emotion in human‐computer interaction, a memory‐based expert system (JANUS; Kearney, 1991) was designed to interpret facial expression in terms of the signaled emotion. Anticipating that a VDU‐mounted camera will eventually supply face parameters automatically, JANUS now accepts manually made measurements on a digitized full‐face photograph and returns emotion labels used by college students. An intermediate representation in terms of face actions (e.g., mouth open) is also used. Production rules convert the geometry into (...) these. A dynamic memory (Kolodner, 1984; Schank, 1982) interprets the face actions in terms of emotion labels. The memory is dynamic in the sense that new emotion labels can be learned with experience. A prototype system has been implemented on a Sun 2/120 system using POPLOG. Validation studies on the prototype suggest that the interpretations achieved are generally consistent with those of college students without formal instruction in emotion signals. (shrink)

Ethical dissonance arises from conflicts between beliefs or behaviors and affects ethical factors such as normality or conformity. This paper proposes a weak signal-oriented framework to investigate ethical dissonance from experiences linked to human–machine interactions. It is based on a systems engineering principle called human-systems inclusion, which considers any experience feedback of weak signals as beneficial to learn. The framework studies weak signal-based scenarios from testimonies of individual experiences and these scenarios are assessed by other (...) people. For this purpose, the framework proposes several databases as sources of weak signals, formalization tools of experience feedback of weak signals, models of references of conformity, ethical factors, and a list of examples of ethical dissonance. It also includes sequential steps to make the latter credible regarding the results from the experimental protocols. The framework was used to investigate ethical dissonance by analyzing experiences pertaining to achieving inclusive mobility. The first example focuses on ethical dissonance in terms of hindrance that goes against autonomous mobility due to a misunderstanding of how the system functions in terms of negative and positive emotions. Two other examples present possible ethical dissonance when the use of safety systems such as car driver-assistance systems, may create danger. Investigating ethical dissonance can then help system-inclusive design or evaluation processes by taking into account scenarios from weak signal-based experiences and making them credible. (shrink)

Consciousness is widely perceived as one of the most fundamental, interesting and difficult problems of our time. However, we still know next to nothing about the relationship between consciousness and the brain and we can only speculate about the consciousness of animals and machines. Human and Machine Consciousness presents a new foundation for the scientific study of consciousness. It sets out a bold interpretation of consciousness that neutralizes the philosophical problems and explains how we can make scientific predictions (...) about the consciousness of animals, brain-damaged patients and machines. Gamez interprets the scientific study of consciousness as a search for mathematical theories that map between measurements of consciousness and measurements of the physical world. We can use artificial intelligence to discover these theories and they could make accurate predictions about the consciousness of humans, animals and artificial systems. Human and Machine Consciousness also provides original insights into unusual conscious experiences, such as hallucinations, religious experiences and out-of-body states, and demonstrates how 'designer' states of consciousness could be created in the future. Gamez explains difficult concepts in a clear way that closely engages with scientific research. His punchy, concise prose is packed with vivid examples, making it suitable for the educated general reader as well as philosophers and scientists. Problems are brought to life in colourful illustrations and a helpful summary is given at the end of each chapter. The endnotes provide detailed discussions of individual points and full references to the scientific and philosophical literature."--Publisher's website. (shrink)

Can autonomous systems replace humans in the performance of their activities? How does the answer to this question inform the design of autonomous systems? The study of technical systems and their features should be preceded by the study of the activities in which they play roles. Each activity can be described by its overall goals, governing norms and the intermediate steps which are taken to achieve the goals and to follow the norms. This paper uses the (...) activity realist approach to conceptualize autonomous systems in the context of human activities. By doing so, it first argues for epistemic and logical conditions that illustrate the limitations of autonomous systems in tasks which they can and cannot perform, and then, it discusses the ramifications of the limitations of system autonomy on the design of autonomous systems. (shrink)

Autonomy has been central to moral and political philosophy for millenia, and has been positioned as a critical aspect of both justice and wellbeing. Research in psychology supports this position, providing empirical evidence that autonomy is critical to motivation, personal growth and psychological wellness. Responsible AI will require an understanding of, and ability to effectively design for, human autonomy (rather than just machine autonomy) if it is to genuinely benefit humanity. Yet the effects on human autonomy (...) of digital experiences are neither straightforward nor consistent, and are complicated by commercial interests and tensions around compulsive overuse. This multi-layered reality requires an analysis that is itself multidimensional and that takes into account human experience at various levels of resolution. We borrow from HCI and psychological research to apply a model (“METUX”) that identifies six distinct spheres of technology experience. We demonstrate the value of the model for understanding human autonomy in a technology ethics context at multiple levels by applying it to the real-world case study of an AI-enhanced video recommender system. In the process we argue for the following three claims: 1) There are autonomy-related consequences to algorithms representing the interests of third parties, and they are not impartial and rational extensions of the self, as is often perceived; 2) Designing for autonomy is an ethical imperative critical to the future design of responsible AI; and 3) Autonomy-support must be analysed from at least six spheres of experience in order to approriately capture contradictory and downstream effects. (shrink)

Deals with topics and insight into the design and architecture of next-generation computing systems that deal with human and social problems. Processor and Internet technologies form the subject matter and focal point, while information and knowledge on the Internet delivered by next-generation mobile networks form the technical core presented.

In sociology, there has been a controversy about whether there is any essential difference between a human being and a tool, or if the tool–user relationship can be defined by co-actor symmetry. This issue becomes more complex when we consider examples of AI and robots, and even more so following progress in the development of various bio-machine hybrid technologies, such as robots that include organic parts, human brain implants, and adaptive prosthetics. It is argued that a concept (...) of autonomous agency based on organismic embodiment helps to clarify the situation. On this view, agency consists of an asymmetrical relationship between an organism and its environment, because the continuous metabolic and regulatory activity of the organism gives rise to its own existence, and hence its specific behavioral domain. Accordingly, most of current technologies are excluded from the class of autonomous agents. Instead, they are better conceptualized as interfaces that mediate our interactions with the world. This has important implications for design: Rather than trying to help humans to achieve their goals by duplicating their agency in artificial systems, it would be better to empower humans directly by enhancing their existing agency and lived experience with technological interfaces that can be incorporated into their embodiment. This incorporation might be especially facilitated by bio-machine hybrid technology that is designed according the principles of biological autonomy and multi-agent coordination dynamics. (shrink)

Robot ethics encompasses ethical questions about how humans should design, deploy, and treat robots; machine morality encompasses questions about what moral capacities a robot should have and how these capacities could be computationally implemented. Publications on both of these topics have doubled twice in the past 10 years but have often remained separate from one another. In an attempt to better integrate the two, I offer a framework for what a morally competent robot would look like and discuss (...) a number of ethical questions about the design, use, and treatment of such moral robots in society. Instead of searching for a fixed set of criteria of a robot’s moral competence I identify the multiple elements that make up human moral competence and probe the possibility of designing robots that have one or more of these human elements, which include: moral vocabulary; a system of norms; moral cognition and affect; moral decision making and action; moral communication. Juxtaposing empirical research, philosophical debates, and computational challenges, this article adopts an optimistic perspective: if robotic design truly commits to building morally competent robots, then those robots could be trustworthy and productive partners, caretakers, educators, and members of the human community. Moral competence does not resolve all ethical concerns over robots in society, but it may be a prerequisite to resolve at least some of them. (shrink)

There is at present a widespread unease about the direction in which our technology is taking us, apparently against our will. Promising advances seem to carry with them unforeseen negative consequences, including damage to the environment and the reduction of work to the trivial mechanical repetition of actions which have no human meaning. However, attempts to design a better, human-centered technology--one that complements rather than rejects human skills--are all too often frustrated by the prevailing belief that (...) "man is a machine," and one, moreover, that compares badly in terms of performance and durability. This contentious and stimulating book offers a new approach, one that refutes four centuries of science based on strictly causal explanations. It shows that man and nature can be viewed as "machines with a purpose," and that the "purpose" can be the advancement of technology to the benefit and not the detriment of the human race and its environment. This fascinating work is accessible to a wide range of readers, scientists and nonspecialists alike. It will interest anyone concerned about the impact of technology and the way it is shaping our world. (shrink)

This paper examines Harold Garfinkel’s work with ELIZA and a related program LYRIC from 1967 to 1969. AI researchers have tended to treat successful human–machine interaction as if it relied primarily on non-human machine characteristics, and thus the often-reported attribution of human-like qualities to communication with computers has been criticized as a misperception—and humans who make such reports referred to as “deluded.” By contrast Garfinkel, building on two decades of prior research on information and communication, (...) argued that the ELIZA and the LYRIC “chatbots” were achieving interactions that felt human to many users by exploiting human sense-making practices. In keeping with his long-term practice of using “trouble” as a way of discovering the taken-for-granted practices of human sense-making, Garfinkel designed scripts for ELIZA and LYRIC that he could disrupt in order to reveal how their success depended on human social practices. Hence, the announcement “Machine Down” by the chatbot was a desired result of Garfinkel’s interactions with it. This early (but largely unknown) research has implications not only for understanding contemporary AI chatbots, but also opens possibilities for respecifying current information systems design and computational practices to provide for the design of more flexible information objects. (shrink)

I will review the main problems concerning commonsense reasoning in machines and I will present resent two different applications - namaly: the Dual PECCS linguistic categorization system and the TCL reasoning framework that have been developed to address, respectively, the problem of typicality effects and the one of commonsense compositionality, in a way that is integrated or compliant with different cognitive architectures thus extending their knowledge processing capabilities In doing so I will show how such aspects are better dealt with (...) at different levels of representation and will discuss how the adoption of a cognitively inspired approach be useful in the design and implementation of the next generation AI systems mastering commonsense. (shrink)

The value sensitive design (VSD) approach to designing transformative technologies for human values is taken as the object of study in this chapter. VSD has traditionally been conceptualized as another type of technology or instrumentally as a tool. The various parts of VSD’s principled approach would then aim to discern the various policy requirements that any given technological artifact under consideration would implicate. Yet, little to no consideration has been given to how laws, regulations, policies and social norms (...) engage within VSD practices. Similarly, how the interactive nature of the VSD approach can, in turn, influence those directives. This is exacerbated when we consider machine ethics policy that have global consequences outside their development spheres. What constructs and models will position AI designers to engage in policy concerns? How can the design of AI policy be integrated with technical design? How might VSD be used to develop AI policy? How might law, regulations, social norms, and other kinds of policy regarding AI systems be engaged within value sensitive design? This chapter takes the VSD as its starting point and aims to determine how laws, regulations and policies come to influence how value trade-offs can be managed within VSD practices. It shows that the iterative and interactional nature of VSD both permits and encourages existing policies to be integrated both early on and throughout the design process. The chapter concludes with some potential future research programs. (shrink)

The implementation of moral decision making abilities in artificial intelligence (AI) is a natural and necessary extension to the social mechanisms of autonomous software agents and robots. Engineers exploring design strategies for systems sensitive to moral considerations in their choices and actions will need to determine what role ethical theory should play in defining control architectures for such systems. The architectures for morally intelligent agents fall within two broad approaches: the top-down imposition of ethical theories, and the (...) bottom-up building of systems that aim at goals or standards which may or may not be specified in explicitly theoretical terms. In this paper we wish to provide some direction for continued research by outlining the value and limitations inherent in each of these approaches. (shrink)

Currently there is a clear trend towards questioning the traditional sovereign human self which for two hundred years has had an undisputed central status within European culture and philosophy. This challenges the tradition of anthropocentrism which in a Scandinavian computer science context has had two theoretical foundations: the workoriented design theory represented by the Scandinavian participatory design philosophy, and the idea of the computer to a rather passive medium for human communication. The process, reducing the computer (...) to a rather passive medium for human communication. The paper firstly examines these two theoretical anthropocentric positions. Secondly it outlines the trend towards challenging the status of the human self within different research contexts. This trend represents a challenge for Human-centred systems design. Finally, it discusses the new demands for conceptualising basic IT research phenomena created by this development, with particular focus on the issue of human-centredness as a systems design strategy. (shrink)

Intelligent algorithms together with various machine learning techniques hold a dominant position among major challenges for contemporary value sensitive design. Self-learning capabilities of current AI applications blur the causal link between programmer and computer behavior. This creates a vital challenge for the design, development and implementation of digital technologies nowadays. This paper seeks to provide an account of this challenge. The main question that shapes the current analysis is the following: What conceptual tools can be developed within (...) the value sensitive design school of thought for evaluating machine learning algorithms where the causal relation between designers and the behavior of their computer systems has been eroded? The answer to this question will be provided through two levels of investigation within the value sensitive design methodology. The first level is conceptual. Within the conceptual level, we will introduce the notion of computer intentionality and will show how this term may be used for solving an issue of non-causal relation between designer and computer system. The second level of investigation is technical. At this level the emphasis will be given to machine learning algorithms. (shrink)

Over the past decades, due to the course towards digitalization of all areas of life, interest in modeling and creating intelligent systems has increased significantly. However, there are now a stagnation in the industry, a lack of attention to analog and bionic approaches as alternatives to digital, numerous speculations on “neuro” issues for commercial and other purposes, and an increase in social and environmental risks. The article provides an overview of the development of artificial intelligence (AI) conceptions toward increasing (...) the human likeness of machines: from the key ideas of A. Turing and J. von Neumann, who initiated the digitalization of society, to discussions about the definition of AI and the emergence of conceptions of strong and weak AI. Special attention is paid to the approach of A. Sloman, to ideas about the architecture and design of complex artificial systems are considered, which make it possible to “emotionally” expand the idea of weak/strong AI. In the article's section on the necessity and possibility of incorporating emotions into the architecture of AI, the authors reveal the goals and methodological limitations for creating an emotional artificial agent. In addition, the article briefly presents the main principles of the authors' architectural approach to the creation of emotional intellectual systems on the example of the cognitive-affective model of architecture, which allow modeling the impact of emotions on the cognitive processes involved in decision-making processes. The described architectural approach to modeling intelligent systems can be used as a conceptual basis for discussing and formulating a strategy for the development of neurocomputing, philosophy of artificial intelligence, and experimental philosophy, for developing innovative research programs, formulating and solving theoretical and methodological problems. (shrink)

We argue that the concept of practical wisdom is particularly useful for organizing, understanding, and improving human-machine interactions. We consider the relationship between philosophical analysis of wisdom and psychological research into the development of wisdom. We adopt a practical orientation that suggests a conceptual engineering approach is needed, where philosophical work involves refinement of the concept in response to contributions by engineers and behavioral scientists. The former are tasked with encoding as much wise design as possible into (...) machines themselves, as well as providing sandboxes or workspaces to help various stakeholders build practical wisdom in systems that are sufficiently realistic to aid transferring skills learned to real-world use. The latter are needed for the design of exercises and methods of evaluation within these workspaces, as well as ways of empirically assessing the transfer of wisdom from workspace to world. Systematic interaction between these three disciplines (and others) is the best approach to engineering wisdom for the machine age. (shrink)

It is a common view that artificial systems could play an important role in dealing with the shortage of caregivers due to demographic change. One argument to show that this is also in the interest of care-dependent persons is that artificial systems might significantly enhance user autonomy since they might stay longer in their homes. This argument presupposes that the artificial systems in question do not require permanent supervision and control by human caregivers. For this reason, (...) they need the capacity for some degree of moral decision-making and agency to cope with morally relevant situations (artificial morality). Machine ethics provides the theoretical and ethical framework for artificial morality. This article scrutinizes the question how artificial moral agents that enhance user autonomy could look like. It discusses, in particular, the suggestion that they should be designed as moral avatars of their users to enhance user autonomy in a substantial sense. (shrink)

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination's complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram and functional near-infrared spectroscopy are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to (...) enhance brain–computer interface performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go. (shrink)

A recurrent theme of human–machine interaction is how interaction is defined and what kind of information is relevant for successful communication. In accordance with the theoretical strategies of social cognition and technical philosophy, we propose a pragmatic interactive approach, to understand the concept of information in human–machine interaction. We start with the investigation of interpersonal interaction and human–machine interaction by concerning triangulation as guiding principle. To illustrate human–machine interaction, we will mainly focus (...) on the interactive relationship between human cognitive skills and their technological environments. In triadic interaction, the concept of information cannot only be understood as a syntactic transformation as in the dyadic relationship. We argue that information as pragmatic action patterns can be understood as physically embodied units that is not purely formal but generated in interactive processes and stabilized in use. By exploring how the concept of affordance is applied in psychology, anthropology and human-centered design, we provide a notion of pragmatic information, focusing on users in the development and use of machines or software systems. The triadic interaction of end user, development team, and product leads to a successful human–machine interaction. This theoretical framework enables us to understand how the new socio-technological environments expand our scope of action and how we form, interpret, and learn to deal with patterns of action among ourselves and with machines. (shrink)

There are plenty of intelligent machines in our world today: digital computers and autonomous robots. At the heart of each of these machines there are automatic formal systems (programs running on a digital computer). Now, if the interpretation of a formal system does not belong to the formal system itself, if the interpretation has to be added, it is worth asking: in the case of these intelligent machines that are massively interspersed in our social interactions, where does the interpretation (...) come from? In this paper, we analyse what we call the invisibility of interpretation. Dealing with various types of formal systems (computers, robots, formalist approaches to Economics), the human source of the interpretation of these systems is sometimes concealed by a formalist restriction. To show how the formalist restriction produces the invisibility of interpretation allows us to underline our responsibility, as human agents, for all this interpretative work—and its importance for us as human beings. (shrink)

The scientific study of living organisms is permeated by machine and design metaphors. Genes are thought of as the ‘‘blueprint’’ of an organism, organisms are ‘‘reverse engineered’’ to discover their func- tionality, and living cells are compared to biochemical factories, complete with assembly lines, transport systems, messenger circuits, etc. Although the notion of design is indispensable to think about adapta- tions, and engineering analogies have considerable heuristic value (e.g., optimality assumptions), we argue they are limited in (...) several important respects. In particular, the analogy with human-made machines falters when we move down to the level of molecular biology and genetics. Living organisms are far more messy and less transparent than human-made machines. Notoriously, evolution is an oppor- tunistic tinkerer, blindly stumbling on ‘‘designs’’ that no sensible engineer would come up with. Despite impressive technological innovation, the prospect of artificially designing new life forms from scratch has proven more difficult than the superficial analogy with ‘‘programming’’ the right ‘‘software’’ would sug- gest. The idea of applying straightforward engineering approaches to living systems and their genomes— isolating functional components, designing new parts from scratch, recombining and assembling them into novel life forms—pushes the analogy with human artifacts beyond its limits. In the absence of a one-to-one correspondence between genotype and phenotype, there is no straightforward way to imple- ment novel biological functions and design new life forms. Both the developmental complexity of gene expression and the multifarious interactions of genes and environments are serious obstacles for ‘‘engi- neering’’ a particular phenotype. The problem of reverse-engineering a desired phenotype to its genetic ‘‘instructions’’ is probably intractable for any but the most simple phenotypes. Recent developments in the field of bio-engineering and synthetic biology reflect these limitations. Instead of genetically engi- neering a desired trait from scratch, as the machine/engineering metaphor promises, researchers are making greater strides by co-opting natural selection to ‘‘search’’ for a suitable genotype, or by borrowing and recombining genetic material from extant life forms. (shrink)

It is highly likely that, to achieve full human–machine symbiosis, truly intelligent cognitive systems—human-like —may have to be developed first. Such systems should not only be capable of performing human-like thinking, reasoning, and problem solving, but also be capable of displaying human-like motivation, emotion, and personality. In this opinion article, I will argue that such systems are indeed possible and needed to achieve true and full symbiosis with humans. A computational cognitive architecture (...) is used in this article to illustrate, in a preliminary way, what can be achieved in this regard. It is shown that Clarion involves complex structures, representations, and mechanisms, and is capable of capturing human cognitive performance as well as human motivation, emotion, personality, and other relevant aspects. It is further argued that the cognitive architecture can enable and facilitate true human–machine symbiosis. (shrink)

The scientific study of living organisms is permeated by machine and design metaphors. Genes are thought of as the ‘‘blueprint’’ of an organism, organisms are ‘‘reverse engineered’’ to discover their functionality, and living cells are compared to biochemical factories, complete with assembly lines, transport systems, messenger circuits, etc. Although the notion of design is indispensable to think about adaptations, and engineering analogies have considerable heuristic value (e.g., optimality assumptions), we argue they are limited in several important (...) respects. In particular, the analogy with human-made machines falters when we move down to the level of molecular biology and genetics. Living organisms are far more messy and less transparent than human-made machines. Notoriously, evolution is an opportunistic tinkerer, blindly stumbling on ‘‘designs’’ that no sensible engineer would come up with. Despite impressive technological innovation, the prospect of artificially designing new life forms from scratch has proven more difficult than the superficial analogy with ‘‘programming’’ the right ‘‘software’’ would suggest. The idea of applying straightforward engineering approaches to living systems and their genomes— isolating functional components, designing new parts from scratch, recombining and assembling them into novel life forms—pushes the analogy with human artifacts beyond its limits. In the absence of a one-to-one correspondence between genotype and phenotype, there is no straightforward way to implement novel biological functions and design new life forms. Both the developmental complexity of gene expression and the multifarious interactions of genes and environments are serious obstacles for ‘‘engineering’’ a particular phenotype. The problem of reverse-engineering a desired phenotype to its genetic ‘‘instructions’’ is probably intractable for any but the most simple phenotypes. Recent developments in the field of bio-engineering and synthetic biology reflect these limitations. Instead of genetically engineering a desired trait from scratch, as the machine/engineering metaphor promises, researchers are making greater strides by co-opting natural selection to ‘‘search’’ for a suitable genotype, or by borrowing and recombining genetic material from extant life forms. (shrink)

Among ethicists and engineers within robotics there is an ongoing discussion as to whether ethical robots are possible or even desirable. We answer both of these questions in the positive, based on an extensive literature study of existing arguments. Our contribution consists in bringing together and reinterpreting pieces of information from a variety of sources. One of the conclusions drawn is that artifactual morality must come in degrees and depend on the level of agency, autonomy and intelligence of the (...) class='Hi'>machine. Moral concerns for agents such as intelligent search machines are relatively simple, while highly intelligent and autonomous artifacts with significant impact and complex modes of agency must be equipped with more advanced ethical capabilities. Systems like cognitive robots are being developed that are expected to become part of our everyday lives in future decades. Thus, it is necessary to ensure that their behaviour is adequate. In an analogy with artificial intelligence, which is the ability of a machine to perform activities that would require intelligence in humans, artificial morality is considered to be the ability of a machine to perform activities that would require morality in humans. The capacity for artificial (artifactual) morality, such as artifactual agency, artifactual responsibility, artificial intentions, artificial (synthetic) emotions, etc., come in varying degrees and depend on the type of agent. As an illustration, we address the assurance of safety in modern High Reliability Organizations through responsibility distribution. In the same way that the concept of agency is generalized in the case of artificial agents, the concept of moral agency, including responsibility, is generalized too. We propose to look at artificial moral agents as having functional responsibilities within a network of distributed responsibilities in a socio-technological system. This does not take away the responsibilities of the other stakeholders in the system, but facilitates an understanding and regulation of such networks. It should be pointed out that the process of development must assume an evolutionary form with a number of iterations because the emergent properties of artifacts must be tested in real world situations with agents of increasing intelligence and moral competence. We see this paper as a contribution to the macro-level Requirement Engineering through discussion and analysis of general requirements for design of ethical robots. (shrink)

Recent philosophy of mind has increasingly focused on the role of technology in shaping, influencing, and extending our mental faculties. Technology extends the mind in two basic ways: through the creative design of artifacts and the purposive use of instruments. If the meaningful activity of technological artifacts were exhaustively described in these mind-dependent terms, then a philosophy of technology would depend entirely on our theory of mind. In this dissertation, I argue that a mind-dependent approach to technology is mistaken. (...) Instead, some machines are best understood as independent participants in their own right, contributing to and augmenting a variety of social practices as active, though often unrecognized, community members. Beginning with Turing’s call for “fair play for machines”, I trace an argument concerning the social autonomy of nonhuman agents through the artificial intelligence debates of the 20th century. I’ll argue that undue focus on the mind has obscured the force of Turing’s proposal, leaving the debates in an unfortunate stalemate. I will then examine a network theoretic alternative to the study of multi-agent complex systems that can avoid anthropocentric, mind-dependent ways of framing human-machine interactions. I argue that this approach allows for both scientific and philosophical treatment of large and complicated sociotechnical systems, and suggests novel methods for designing, managing, and maintaining such systems. Rethinking machines in mind-independent terms will illuminate the nature, scope, and evolution of our social and technological practices, and will help clarify the relationships between minds, machines, and the environments we share. (shrink)

Replication or even modelling of consciousness in machines requires some clarifications and refinements of our concept of consciousness. Design of, construction of, and interaction with artificial systems can itself assist in this conceptual development. We start with the tentative hypothesis that although the word “consciousness” has no well-defined meaning, it is used to refer to aspects of human and animal informationprocessing. We then argue that we can enhance our understanding of what these aspects might be by designing (...) and building virtual-machine architectures capturing various features of consciousness. This activity may in turn nurture the development of our concepts of consciousness, showing how an analysis based on information-processing virtual machines answers old philosophical puzzles as well enriching empirical theories. This process of developing and testing ideas by developing and testing designs leads to gradual refinement of many of our pre-theoretical concepts of mind, showing how they can be construed as implicitly “architecture-based” concepts. Understanding how humanlike robots with appropriate architectures are likely to feel puzzled about qualia may help us resolve those puzzles. The concept of “qualia” turns out to be an “architecture-based” concept, while individual qualia concepts are “architecture-driven”. (shrink)

Resumen: El presente trabajo analiza si los Sistemas de apoyo a la decisión (DSSs) y otros asistentes para su uso, como las Reflection machines o los Personal Assistants that Learn (PAL), contribuyen de hecho a una mejora cognitiva, como habitualmente se tiende a asumir. Es decir, se examina si su potencial para expandir e impulsar la acción de las facultades cognoscitivas se ve efectivamente actualizado y, en consecuencia, si sirven para reafirmar el sentido capacitante de la IA y la extensión (...) de la agencia del sujeto a través de ella. Para ello, se pone de manifiesto que el diseño y aplicación de estos sistemas están mediados por una comprensión del conocimiento que actúa a modo de marco que limita sus posibilidades de mejorar las capacidades cognoscitivas humanas. Se analizan las condiciones en que la mejora cognitiva es posible a través del uso de estos sistemas, que remiten a una aproximación integradora que no tiene en cuenta solo el factor tecnológico, sino que repara en cómo, en su uso, intervienen factores gnoseológicos, así como psicológicos y sociológicos, que median en la capacidad efectiva de estos sistemas de ampliar las facultades de conocimiento, y de los que las tecnologías se sirven para ejercer su influencia. Abstract: The present article questions whether Decision Support Systems (DSSs) and other assistants for their use, such as Reflection machines or Personal Assistants that Learn (PAL), do in fact contribute to cognitive enhancement, as it is usually taken for granted. In other words, it is analysed whether their potential to effectively expand or augment cognitive capacities is actualised and, therefore, if the capabilities and the agency of the person appear in fact extended through artificial intelligence. For that purpose, it is highlighted that the design and implementation of AI systems are dependent on a certain understanding of human knowledge that acts as a framework that limits their actual capacity to enhance the cognitive faculties of the mind. The conditions under which this cognitive enhancement is possible through these systems are analysed and require an integrative approach that not only takes technology into account but realises that, in its use, gnoseological, as well as psychological and sociological factors are involved and mediate in the actual capacity of these systems to extend cognitive functions. These factors are also crucial for these technologies to exert their influence. (shrink)

Human Specialization in Design and Technology explores emerging trends in learning and training-standardization, customization, personalization-with a unique focus on human needs and conditions. Analyzing evidence from current academic research as well as the popular press, this concise volume defines and examines the trajectory of instructional design and technologies toward more human-centered and specialized products, services, processes, environments, and systems. Examples from education, healthcare, business, and other sectors offer real-world demonstrations for scholars and graduate students (...) of educational technology, instructional design, ICT, and business development. (shrink)

The human-centred debate in Britain focuses on the idea of human-machine symbiosis, and the “Dialogue” debate in Scandinavia focuses on the deep understanding of human communication, through a process of inner reflection. Both of these debates provide a framework for the participatory design of AI systems.The emergence of “social Europe” creates the desirability for a sharing of social and cultural knowledge and resources among the citizens of Europe. This raises the possibility of exploiting the (...) potential of new technology for the distribution and transfer of knowledge and skill among the citizens of Europe whilst retaining the diversity of their cultures and languages. These issues of transfer have been the focus of many AI researchers who have been concerned with the design of AI systems for the education, training and welfare of disadvantaged citizens. It is suggested that central to such a transfer is the design of participatory learning systems which focus on the mediation and distribution of knowledge in diverse socio-cultural contexts.In this paper, the SEAKE Centre's projects, especially “Brighton Parosi”, and “Adult Literacy” are discussed to provide an insight into the concepts and practice of designing such participatory systems. The philosophy of these projects was rooted in the sharing of social and cultural experiences, through participatory learning. The development of the projects involved a process of exchange of language and culture in both formal and informal learning environments. For the Parosi Project social interaction was the central core for the exchange of cultural experiences and the language acquisition process. The Adult Literacy project built on the exchange process, and used the idea of the mediation of knowledge for the design of a knowledge based system. The design process developed by the above two projects could be described as a participatory design approach which builds upon and extends the human-machine symbiosis and dialogue approaches. (shrink)

Value Sensitive Design (VSD) is an established method for integrating values into technical design. It has been applied to different technologies and, more recently, to artificial intelligence (AI). We argue that AI poses a number of challenges specific to VSD that require a somewhat modified VSD approach. Machine learning (ML), in particular, poses two challenges. First, humans may not understand how an AI system learns certain things. This requires paying attention to values such as transparency, explicability, and (...) accountability. Second, ML may lead to AI systems adapting in ways that ‘disembody’ the values embedded in them. To address this, we propose a threefold modified VSD approach: 1) integrating a known set of VSD principles (AI4SG) as design norms from which more specific design requirements can be derived; 2) distinguishing between values that are promoted and respected by the design to ensure outcomes that not only do no harm but also contribute to good; and 3) extending the VSD process to encompass the whole life cycle of an AI technology in order to monitor unintended value consequences and redesign as needed. We illustrate our VSD for AI approach with an example use case of a SARS-CoV-2 contact tracing app. (shrink)

Autonomous weapons systems, often referred to as ‘killer robots’, have been a hallmark of popular imagination for decades. However, with the inexorable advance of artificial intelligence systems (AI) and robotics, killer robots are quickly becoming a reality. These lethal technologies can learn, adapt, and potentially make life and death decisions on the battlefield with little-to-no human involvement. This naturally leads to not only legal but ethical concerns as to whether we can meaningful control such machines, and if (...) so, then how. Such concerns are made even more poignant by the ever-present fear that something may go wrong, and the machine may carry out some action(s) violating the ethics or laws of war. -/- Researchers, policymakers, and designers are caught in the quagmire of how to approach these highly controversial systems and to figure out what exactly it means to have meaningful human control over them, if at all. -/- In Designed for Death, Dr Steven Umbrello aims to not only produce a realistic but also an optimistic guide for how, with human values in mind, we can begin to design killer robots. Drawing on the value sensitive design (VSD) approach to technology innovation, Umbrello argues that context is king and that a middle path for designing killer robots is possible if we consider both ethics and design as fundamentally linked. Umbrello moves beyond the binary debates of whether or not to prohibit killer robots and instead offers a more nuanced perspective of which types of killer robots may be both legally and ethically acceptable, when they would be acceptable, and how to design for them. (shrink)

An emerging discourse about automated agricultural machinery imagines farms as places where farmers and workers do not need to be, but also implicitly frames farms as intolerable places where people do not want to be. Only autonomous machines, this story goes, can relieve farmers and workers of this presumed burden by letting them ‘farm at a distance’. In return for this distanced autonomy, farmers are promised increased control over their work-life balance and greater farm productivity from letting ‘smart’ robots assume (...) control over the operational environment. Drawing upon the ways that these machines are promoted by manufacturers in various media, we trace the nascent contours of what we term a liberatory sociotechnical imaginary for agricultural automation across three cases—automated milking systems, self-driving tractors, and robotic strawberry pickers. We show how promises for new freedoms and autonomy are flexibly deployed to respond to distinct frictions that farmers, workers, and even farm animals experience in different modes of industrial agriculture. However, underlying these promises is the purposefully understated self-interest of manufacturers, who stand to gain further control over farms if automated technologies assume a central role in agriculture. Through the liberatory rhetoric, we contend, the imaginary seeks to enroll farmers into a socio-technical network that creates new relations of dependence upon the companies who design, sell, maintain, and often retain ownership over automated technologies. While potentially powerful, this imaginary may nonetheless fail to coalesce as farmers, workers, and agroecosystems exert their own agency on automated imaginaries and technological futures for agriculture. (shrink)

Wendell Wallach and Colin Allen’s Moral Machines: Teaching Robots Right From Wrong (Oxford University Press, 2009) explores efforts to develop machines that, not only can be employed for good or bad ends, but which themselves can be held morally accountable for what they do— artificial moral agents (AMAs). This essay is a critical response to Wallach and Allen’s conjectures. Although Wallach and Allen do not suggest that we are close to being able to create full-fledged AMAs, they do talk seriously (...) about making incremental progress in the direction of creating them (even if we never fully succeed). However, there are important questions about the moral development of AMAs that Moral Machines does not address. Given the responsibilities entrusted to human moral agents, we take questions about their moral development very seriously. In the case of children, the hope is that eventually they will develop into full-fledged moral agents. How might we expect this to go with less than fully formed AMAs? Will there be a comparable story of moral development and moral education that we can tell? (shrink)

As the use of AI systems continues to increase, so do concerns over their lack of fairness, legitimacy and accountability. Such harmful automated decision-making can be guarded against by ensuring AI systems are contestable by design: responsive to human intervention throughout the system lifecycle. Contestable AI by design is a small but growing field of research. However, most available knowledge requires a significant amount of translation to be applicable in practice. A proven way of conveying (...) intermediate-level, generative design knowledge is in the form of frameworks. In this article we use qualitative-interpretative methods and visual mapping techniques to extract from the literature sociotechnical features and practices that contribute to contestable AI, and synthesize these into a design framework. (shrink)

In this paper we review some problems with traditional approaches for acquiring and representing knowledge in the context of developing user interfaces. Methodological implications for knowledge engineering and for human-computer interaction are studied. It turns out that in order to achieve the goal of developing human-oriented (in contrast to technology-oriented) human-computer interfaces developers have to develop sound knowledge of the structure and the representational dynamics of the cognitive system which is interacting with the computer.We show that in (...) a first step it is necessary to study and investigate the different levels and forms of representation that are involved in the interaction processes between computers and human cognitive systems. Only if designers have achieved some understanding about these representational mechanisms, user interfaces enabling individual experiences and skill development can be designed. In this paper we review mechanisms and processes for knowledge representation on a conceptual, epistemological, and methodologieal level, and sketch some ways out of the identified dilemmas for cognitive modeling in the domain of human-computer interaction. (shrink)

Accountability and responsibility are key concepts in the academic and societal debate on Autonomous Weapon Systems, but these notions are often used as high-level overarching constructs and are not operationalised to be useful in practice. “Meaningful Human Control” is often mentioned as a requirement for the deployment of Autonomous Weapon Systems, but a common definition of what this notion means in practice, and a clear understanding of its relation with responsibility and accountability is also lacking. In this (...) paper, we present a definition of these concepts and describe the relations between accountability, responsibility, control and oversight in order to show how these notions are distinct but also connected. We focus on accountability as a particular form of responsibility—the obligation to explain one’s action to a forum—and we present three ways in which the introduction of Autonomous Weapon Systems may create “accountability gaps”. We propose a Framework for Comprehensive Human Oversight based on an engineering, socio-technical and governance perspective on control. Our main claim is that combining the control mechanisms at technical, socio-technical and governance levels will lead to comprehensive human oversight over Autonomous Weapon Systems which may ensure solid controllability and accountability for the behaviour of Autonomous Weapon Systems. Finally, we give an overview of the military control instruments that are currently used in the Netherlands and show the applicability of the comprehensive human oversight Framework to Autonomous Weapon Systems. Our analysis reveals two main gaps in the current control mechanisms as applied to Autonomous Weapon Systems. We have identified three first options as future work for the design of a control mechanism, one in the technological layer, one in the socio-technical layer and one the governance layer, in order to achieve comprehensive human oversight and ensure accountability over Autonomous Weapon Systems. (shrink)

This book brings together philosophical approaches to cooperation and collective agency with research into human-machine interaction and cooperation from engineering, robotics, computer science and AI. Bringing these so far largely unrelated fields of study together leads to a better understanding of collective agency in natural and artificial systems and will help to improve the design and performance of hybrid systems involving human and artificial agents. Modeling collective agency with the help of computer simulations promises (...) also philosophical insights into the emergence of collective agency. The volume consists of four sections. The first section is dedicated to the concept of agency. The second section of the book turns to human-machine cooperation. The focus of the third section is the transition from cooperation to collective agency. The last section concerns the explanatory value of social simulations of collective agency in the broader framework of cultural evolution. (shrink)