An increasingly popular suggestion is that empathy and/or sympathy plays a foundational role in understanding harm norms and being motivated by them. In this paper, I argue these emotions play a rather more moderate role in harms norms than we are often led to believe. Evidence from people with frontal lobe damage suggests that neither empathy, nor sympathy is necessary for the understanding of such norms. Furthermore, people's understanding of why it is wrong to harm varies and is by no means limited to considerations of welfare arising from the abilities to sympathize and/or empathize. And the sorts of considerations of welfare that are central to sympathy and, to some extent empathy, are often already moralized. As such, these considerations cannot form the non-moral foundation of harm norms. Finally, empathy and sympathy are not the only emotions that motivate harm norms. Indeed, much of the evidence that has been adduced in favor of the motivational force of empathy and sympathy are studies on helping, which is quite a different behavior than aggression inhibition. Understanding and being motivated by harm norms are complex abilities. To understand them better, we need to move beyond the current fixation on empathy and sympathy.