For decades, the field of system safety has designed safe systems by reducing the risk of physical harm to humans, property and the environment to an acceptable level. Recently, this definition of safety has come under scrutiny by governments and researchers who argue that the narrow focus on reducing physical harm, whilst necessary, is not sufficient to secure the safety of AI systems. There is growing pressure to expand the scope of safety in the context of AI to address emerging (...) harms, with particular emphasis being placed on the ways AI systems can reinforce and reproduce systemic harms. In this paper, we advocate for expanding the scope of conventional safety to include non-physical harms in the context of AI. However, we caution against broadening the scope to address systemic harms, as doing so presents intractable practical challenges for current safety methodologies. Instead, we propose that the scope of safety-related harms should be expanded to include psychological harms. Our proposal is partly motivated by the debates and evidence on social media, which fundamentally reshaped how harm is understood and addressed in the digital age, prompting new regulatory frameworks which aimed to protect users from the psychological risks of the technology. We draw on this precedent to motivate the inclusion of psychological harms in AI safety assessments. By expanding the scope of AI safety to include psychological harms, we take a critical step toward evolving the discipline of system safety into one that is better tuned and equipped to protect users against the complex and emerging harms propagated by AI systems. (shrink)

With recent advancements in systems engineering and artificial intelligence, autonomous agents are increasingly being called upon to execute tasks that have normative relevance. These are tasks that directly—and potentially adversely—affect human well-being and demand of the agent a degree of normative-sensitivity and -compliance. Such norms and normative principles are typically of a social, legal, ethical, empathetic, or cultural nature. Whereas norms of this type are often framed in the abstract, or as high-level principles, addressing normative concerns in concrete applications of (...) autonomous agents requires the refinement of normative principles into explicitly formulated practical rules. This paper develops a process for deriving specification rules from a set of high-level norms, thereby bridging the gap between normative principles and operational practice. This enables autonomous agents to select and execute the most normatively favourable action in the intended context premised on a range of underlying relevant normative principles. In the translation and reduction of normative principles to SLEEC rules, we present an iterative process that uncovers normative principles, addresses SLEEC concerns, identifies and resolves SLEEC conflicts, and generates both preliminary and complex normatively-relevant rules, thereby guiding the development of autonomous agents and better positioning them as normatively SLEEC-sensitive or SLEEC-compliant. (shrink)

Policymakers and researchers consistently call for greater human accountability for AI technologies. We should be clear about two distinct features of accountability.