Moral Responsibility for Harm Caused by Computer System Failures

Abstract

When software is written and then utilized in complex computer systems, problems often occur. Sometimes these problems cause a system to malfunction, and in some instances such malfunctions cause harm. Should any of the persons involved in creating the software be blamed and punished when a computer system failure leads to persons being harmed? In order to decide whether such blame and punishment are appropriate, we need to first consider if the people are “morally responsible”. Should any of the people involved in creating the software be held morally responsible, as individuals, for the harm caused by a computer system failure?

This article provides one view of moral responsibility and then discusses some barriers to holding people morally responsible. Next, it provides information about the Therac-25, a computer-controlled medical linear accelerator, and its computer systems failures that led to deaths and injuries. Finally it investigates whether two key people involved in the Therac-25 case could reasonably be considered to have some degree of moral responsibility for the deaths and injuries. The conclusions about whether or not these people were morally responsible necessarily rest upon a certain amount of speculation about what they knew and what they did. These limitations, however, should not cause us to conclude that discussions of moral responsibility are fruitless. In some cases, determinations of moral responsibility may be made and in others the investigation is still worthwhile, as the article demonstrates.