Introduction

It is estimated that one in three women experience intimate partner violence (IPV) across the course of their life (World Health Organization, 2021).Footnote 1 IPV violates the rights of victim-survivors and their children as well as threatening their safety. Around the world, approximately 137 women are killed by a current or former intimate partner or family member every day (UNODC, 2018). “Security” is a perennial justification for increasingly comprehensive forms of surveillance. It is little surprise, then, that calls have arisen to deploy “artificial intelligence” (AI) as a means of addressing IPV (Bland, 2020).Footnote 2 Enthusiasm for the use of AI to combat IPV is continuous with broader developments in the realm of “data-driven” policing, which relies on large scale data collection, combined with the use of inferential AI systems, to discern patterns that indicate criminal activity, past, present, or future (Andrejevic, 2017; Brayne, 2020).

An obvious challenge when it comes to the use of AI to protect women from IPV is the fact that such violence often occurs in the home, where, for instance, public surveillance cameras are not able to detect it. However, the advent of the “internet of things” and, in particular, the commercial success and popular uptake of “smart speakers” powered by sophisticated AI means that intelligent surveillance of the domestic environment is increasingly possible. The idea that smart speakers might be used to combat IPV also tends to arise naturally in the context of an important set of criticisms of these devices, which highlight their capacity to be used to as tools of coercive control and/or emotional abuse (Bowles, 2018).Footnote 3 The suggestion that smart speakers could work to protect women against IPV has obvious appeal as a rhetorical strategy to resist criticisms of this technology. Correspondingly, there are now various proposals to use AI and smart speakers to detect or report, or otherwise attempt to protect women from, IPV—an approach that might be called “embracing ‘Big Sister’”.Footnote 4

In this paper, we clarify what might be possible using existing or near-term technology when it comes to detecting and/or reporting IPV using smart speakers and also begin the project of evaluating this embrace of “Big Sister”, both ethically and politically. In the first section, ““Big Sister” as a solution to IPV?”, we outline the prima facie case that smart speakers might be deployed to combat IPV. The second section, “What to detect?”, distinguishes between various phenomena that smart speakers might be designed to detect in the service of this goal. The third section, “Possible responses on detecting the target state of affairs”, discusses the various actions that smart speakers might be programmed to perform if they detect any of these states of affairs. In the fourth section, “Two challenges”, we discuss one ethical and political challenge and one technical challenge to the use of smart speakers to detect and report IPV. The fifth section, “Social and ethical implications”, sets out, and critically evaluates, the social and ethical implications of the proposed technology and argues that they establish a strong prima facie case against developing it. Before we embrace Big Sister as a solution to IPV we should think carefully about the consequences of doing so. In the sixth section, “Ethical decision points”, we point out that the ethical landscape looks very different depending on whether one is considering the decision to develop the technology or the decision to use it once it has been developed. We conclude that if, as we suggest they should, activists and governments wish to avoid the privatisation of responses to IPV, and/or opening the door to even more ubiquitous surveillance of domestic spaces, with unpredictable consequences in the longer term, and/or increasing the risk posed to members of minority communities by police responses to IPV, and/or the danger that more powerful smart speakers will be co-opted by men to control and abuse women, then they should campaign to discourage the development of this technology rather than wait until these systems are developed before resisting them. If it is, nevertheless, judged that the moral urgency of the problem of IPV justifies exploring what might be possible by developing this technology, even in the face of these risks, then it will be imperative that victim-survivors from a range of community cohorts and demographics, as well as government and non-government stakeholders, are engaged in the realization of this technology and in the formulation of the legislation and policies that will be needed to regulate it.

“Big Sister” as a solution to IPV?

“Smart speakers”, such as Amazon’s Echo (with “Alexa”), Apple’s Homepod (with “Siri”), and Google’s Nest (with Google Assistant), are always-on audio devices that allow users to play music, request information from the internet and, in some cases, control other networked household devices (such as climate control, lighting, and energy use) by means of voice prompts interpreted by AI. According to Edison Research, as of 2021, a quarter of Australians, and one-third of American households, had at least one smart speaker (Edison Research, 2021). About one-quarter of households with smart speakers as of 2021 had three or more installed in the home (Edison Research, 2021). Market research indicates that the primary location of the speakers is in common living areas such as living rooms and kitchens, but the next most common location is the main bedroom (Edison Research, 2021).

The number of speakers in the community and their capacity to detect audio signals from multiple spaces in a household means that they offer an unprecedented opportunity to determine what is going on in domestic spaces in real time. Engineers at Google are already envisioning a future in which “always on” smart speakers equipped with a growing array of sensors (including infrared heat detectors, vapor detectors, microphones, and smart cameras) are able to provide an increasingly comprehensive data portrait of the household, which in turn can be used to make inferences and predictions about the activities of inhabitants. A patent submitted by Google for a smart home control system anticipates that:

Certain contextual queues [sic] of individual occupants may be used to infer the emotional state of household occupants. For example, crying may signify a sad emotional state, whereas laughing may signify a happy emotional state … audio queues [sic], such as audio signatures of crying, laughing, elevated voices, etc. may be used to infer emotions. Further, infrared information, such as body temperature, etc. may be used in an emotional state inference. Further, some general cultural norms, which can often be keyed to household location, may be used in universally inferring an emotional state of household occupants (Fadell et al., 2018: p. 30).

This patent goes on to outline an array of possible uses of “smart home automation systems” to implement and oversee household policies, ranging from restricting internet and video game access, to making sure occupants turn the water off when they are brushing their teeth, to monitoring “undesirable” activities such as the use of “foul language” (Fadell et al., 2018: p. 30).

One of the uses envisioned in this—admittedly speculative—document is the deployment of smart speakers for detecting the threat of possible “mischief” via both audio and location data in the household as well as various other contextual and cultural cues. As the patent puts it,

…particular activities or other context may be used to infer that mischief is occurring, or to exclude an inference that mischief is occurring.

The examples in the patent refer to unattended children getting into trouble, but the stated goal of being able to detect both harm in the household and the real-time emotional state of its inhabitants suggest IPV is something smart speakers might be set to detect. In fact, a team of researchers at the University of Missouri (Shah et al., 2018: p. 5181) has already begun developing an AI system for detecting and reporting IPV. In a paper describing this project they report that:

In this paper, we have proposed a solution, called violence detection to tackle violence issues and make a successful smart home violence detection system. Our main contribution is divided into two main parts. In the first part of the paper, we have created a model for audio mainly violence detection using IoT sensors. Based on this audio violence data, it has been observed that data analytics to predict types of domestic violence is required in home, where people are unaware of the surrounding happenings. We have put emphasis on violence data such as screaming, siren, explosion, gunshot and glass breaking that may happen when all neighbors are uninformed of the situation. The second part of our contribution is to design an automated detection system for domestic violence in such a way that if the users are away or even sleeping the system will have the capability to inform the police department for possible actions such as in a situation of screaming. We have used machine learning approach [sic] to design a system to recognize the sound occurring and inform the police department when any suspicious sound happening around is detected. The focus of machine learning tasks is towards detection of domestic violence sounds such as gunshot, screaming, glass breaking, explosion and siren [sic].

Muraleedharan and Garcia-Constantino (2022: p. 358) suggest that:

the use of audio sensors … would detect cases of domestic violence in a more reliable way than using other types of sensors such as accelerometers, thermal or radar. Different types of human activity related to domestic violence such as shouting, crying, screaming and chatting, can be effectively detected using audio sensors compared to other types of sensors

and have developed a prototype system for “Domestic Violence Detection Using Smart Microphones”.

Given the (purportedly) feminist motivations of this resort to ubiquitous surveillance to combat IPV, it does not seem inappropriate to refer to it as a turn to “Big Sister” rather than “Big Brother”. In so doing, we do not seek to equate the attempt to address IPV with authoritarian control; rather, we note that, as surveillance scholar David Lyon has highlighted in his writing on surveillance, the irony in the original “Big Brother” epithet results from the fact that surveillance can operate in the register of care: the use of smart speakers to detect and/or report IPV would be an example of what Lyon calls “watching-to-care-for” (Bauman & Lyon, 2012: p. 87).Footnote 5

Our goal in what follows is to survey and critically assess the ethical and political questions that arise regarding any embrace of Big Sister.Footnote 6 Before we can discuss the ethical and political issues, though, we must first clarify what smart speakers might be able to detect (the section entitled “What to detect?”) and also what they might do when they have detected some target set of affairs (in the section headed “Possible responses on detecting the target state of affairs”).

What to detect?

The capacity of smart speakers to detect particular states of affairs is, obviously, a function of precisely what it is that they are trying to detect. In this section, we survey various possible targets for detection and offer some initial discussion of the likelihood that smart speakers might be able to detect them.Footnote 7 Our discussion is ordered, for the most part, from the least to the most speculative possibilities. It is important to emphasise that we are not here endorsing, or even attempting an all-things-considered evaluation of, the project of setting out to detect any of these states of affairs: we leave discussion of the merits of the project for a subsequent section.

The first possibility, which existing smart speakers already have the capacity to realise, is to recognise an explicit call for help from a victim of an assault in progress. If someone says “Siri, call emergency services” an Apple Homepod or HomePod Mini will attempt to do so (Apple, 2023; Writtenhouse, 2021). In some countries, and with some—but not all—telecommunications providers, Google’s Nest Audio device is also capable of calling the local number for the emergency services on command (Google Nest Help, 2023; Telstra, 2023).Footnote 8 Amazon’s “Alexa” is not able to call the emergency services but can call a particular number, designated by the user as an emergency contact, if one says, for instance, “call for help” (Amazon, 2023). Most of these devices will also allow users to set up bespoke commands to call emergency services, or designated emergency contacts, so that they can call for help without necessarily alerting an assailant that they have done so.

A second possibility is that smart speakers might be trained to detect physical assaults in the absence of an explicit instruction to call for help (Shah et al., 2018). Gunshots (Gerosa et al., 2007), screams (Huang et al., 2010; Lei & Mak, 2016), shouting (Atrey et al., 2006; Pohjalainen et al., 2011), crying, crashing and thumping noises, or particular combinations thereof, might be taken to indicate that a violent incident is in progress.Footnote 9 Generating a dataset to train a machine learning system on so that it can detect assaults in progress would pose significant challenges, including ethical challenges associated with consent to the use of data to this purpose. Nevertheless, it might be possible to do so by combining audio from existing smart speaker systems and/or recordings of calls to emergency services, with police incident reports. It is unlikely that any such system could detect every assault, not least because not every assault is reported to the police, which means that the training dataset would contain significant biases. Despite these challenges, we suspect that detecting a significant percentage of assaults-in-progress will be well within the capability of smart speaker systems in the not-too-distant future. Additionally, since users often connect their smartphones to their speakers, it is possible that the “accident” or “crash” detection sensors on a phone or smart watch could be used to signal an alert in the case of physical assault.

A third thing that smart speaker systems might be trained to detect is explicit verbal threats of violence. Again, we suspect that it is already within the capacity of existing smart speakers to recognise sentences or sentence fragments that constitute explicit threats, albeit at the cost of a significant numbers of false negatives and false positives (we discuss the significance of this qualification below). Importantly, there are at least two different reasons that might be given to develop and/or deploy this capacity. Such threats are themselves crimes that cause significant harms to their victim-survivors. However, they are also highly correlated with physical violence both in the immediate and in the longer term (Monkton-Smith et al., 2014). That is to say, detecting these verbal threats might allow a smart speaker to determine when someone was at an elevated risk of assault in the future.

A fourth thing that smart speaker systems might be trained to detect is emotional abuse in the absence of physical violence or explicit verbal threats of violence. Characterising and identifying such patterns is likely to be tricky, but not, we think, impossible: one way to do this would be to search for correlations with the other sorts of incidents mentioned above. Again, this project might be motivated by the desire to prevent the harm of this abuse itself or to avert future harm associated with other crimes that often stem from, or accompany, emotional abuse.

A fifth target might be gendered patterns of social relations that do not constitute emotional abuse but occur in the context of IPV and/or are predictive of the other harms mentioned above. For instance, perhaps it would be possible for smart speakers to detect patterns of household behaviour that constitute “coercive control” or are highly gendered to the point of gender injustice: there is compelling evidence that the former can be associated with fatal violence (Johnson et al., 2019); the latter are bad for women—arguably by definition—and are strongly predictive of gendered violence (World Health Organization, 2021).

A sixth target might be other patterns of behaviour that are known to be associated with risks to women via IPV, such as alcohol and drug abuse, and financial stress (Capaldi et al., 2012). Again, we expect that it will be possible to identify at least some of these from household audio data, especially if the smart speaker is also able to supplement this data with online data about household members.

Finally—and this is the most speculative possibility of all—the algorithms used in smart speaker systems might be trained to identify other signals in audio data that are correlated with violence or other harms to women in the future despite not being currently recognised as such. The promise of big data and unsupervised deep learning is that it can identify patterns that human beings cannot (Mayer-Schönberger & Cukier, 2013; Zarsky, 2012). Given access to a sufficiently large dataset, which included recorded audio from households, as well as records of visits by the police and social services and of relationship breakdowns, it is possible that a machine learning algorithm could identify signals in the audio data that are predictive of gendered violence, either physical or emotional, despite falling into none of the categories above. These would be of especial interest given that they might make it possible to alert women, or relevant authorities or social services, to risks of which they might otherwise be unaware. It can be difficult to predict individual behaviour based on data analysis but high incidence events tend to be easier to predict than exceptional ones. Unfortunately, IPV is widespread and endemic, which means that the potential exists to collect large amounts of data, which would, in turn support more robust predictions.

It must be acknowledged that there are significant technical and ethical challenges associated with developing the capacity to detect most of the states of affairs discussed here. In particular, training algorithms to detect them would require that sufficient numbers of people were willing to make audio recorded by smart speaker systems available to researchers and also to donate data relating to their experiences of assault, verbal threats, emotional abuse, coercive control, and other forms of harm. That being said, we do not think it is impossible that a community of motivated research participants might be mobilised to help develop this technology by doing so. Moreover, as is the case with many applications of AI, once a sufficient number of people began deploying this technology, its performance could be expected to improve rapidly.

Possible responses on detecting the target state of affairs

As well as there being several different states of affairs that smart speakers might be enabled to detect, there are a number of different actions that might be taken by the speakers once they have detected a given state of affairs. The appropriateness of these actions will depend, at least in part, on the harm associated with the condition detected itself and the risk of future harms with which it is associated. Again, our purpose in this section is to identify, rather than to attempt an all-things-considered evaluation of, the various options: we leave the latter task to discussion in the following sections.

The first, and the most obvious, thing a smart speaker might do is alert police to the existence of some relevant state of affairs. This is, presumably, only appropriate where an individual instructs the smart speaker to call the police, either overtly or covertly, or where a smart speaker detects a physical assault in progress or, perhaps, an explicit verbal threat of violence.

The case for calling the police when a physical assault in progress has been detected is—at least superficially—strong (Zenor, 2020). Indeed, as we shall discuss further below, once a device has the capacity to detect an assault in progress it is difficult to see how designers could justify it doing anything else. However, as we shall also discuss further below, this prospect raises difficult problems related to the relative occurrence of false positives and false negatives in the performance of the algorithm that enables the detection of assaults.

Insofar as explicit verbal threats of violence are themselves serious criminal offences, a case can be made for smart speaker systems calling the police when these occur. The strength of this case will depend upon how strong the association is between such threats and physical assaults shortly afterwards. If calling the police is unlikely to avert a significant number of physical assaults (noting that the operationalisation of this test is likely to be highly controversial) then an alternative might be to allow the target of the threats to send a recording of the threats to the police at a later date.

An important further question in relation to these three cases is whether the fact that the speaker has taken some action, and the nature of that action, is available to the target and/or initiator of an assault or threat (Zenor, 2020: p. 861). Informing the individual being assaulted or threatened that help is on its way may help them act to promote their own interests. However, it may be difficult to notify the target of an assault/threat that the police have been called without risking notifying the person carrying out the assault or making the threat. It is possible that informing the latter that a call had been made to the police might have a deterrent effect when it comes to the continuation of an assault or carrying out a threat that has been made. However, it is also possible that it might exacerbate the situation and/or impact on the relationship between the parties in such a way as to increase the likelihood of future violence (Buxton-Namisnyk, 2022; Dichter & Gelles, 2012; Richie, 2012). The inability of AI systems to make the contextual judgements that a (human) bystander in the same position has the capacity to make might be argued to be a significant limitation of such systems—and perhaps even a reason for eschewing them. However, it might also be argued that the fact that smart speakers can simply be set to follow best practice in relation to whether to call the police when an assault or threat is detected is a virtue of the technology.

Alerting the police is the most immediate and high stakes action that a smart speaker might take. Where there is no imminent risk of harm a wider range of options becomes plausible. In households where emotional abuse, coercive control, or other patterns of behaviour that are associated with harms to women, are present, smart speakers might: alert social services other than the police; alert the woman to the risk that she faces (Zenor 2020: p. 862); provide information about relevant options, and support services, available to the woman; and/or recommend that the potential perpetrator seek out and/or be offered treatment. Data, including recordings, from smart speakers might serve as documentation of crimes, or patterns of behaviour, by perpetrators, for the purposes of various institutional responses to these, including apprehended violence orders, or criminal prosecutions. Absent an immediate threat to the well-being of a vulnerable party, it is plausible to think that the woman herself should determine the action the speaker will take in response to detecting various states of affairs via settings providing an appropriate list of options.

Two challenges

Any attempt to use smart speakers in any of the roles we have described above would generate a number of challenges, including practical, ethical, and political challenges. For the most part, we leave discussion of the ethical and political challenges for the sections entitled “Social and ethical implications” and “Two challenges” below. However, there is one ethical/political challenge, which also has a practical component, and one practical challenge, which also has an ethical/political component, that would need to be overcome before any of the other challenges would arise, which we therefore discuss here.

The ethical/political challenge concerns the role of smart speakers in households in which there is a significant risk of IPV. In order for it to be plausible that these might serve the interests of women in these households, it would need to be the case that those who were concerned about the possibility of such violence had access to the relevant settings of the device that determine whether it will detect any of the states of affairs described above, which states of affairs it would detect, and how it would respond.

There are at least three reasons to think that this is likely to pose a significant barrier to the plausibility of this approach to the problem of IPV. First, information technologies like smart speakers are already strongly “gendered” in ways that tend to make it less likely for women to develop mastery (!) of them or to see them as potential material allies (Harris & Vitis, 2020). Even in households that aren’t obviously structured by (other forms of) gendered injustice, smart speaker systems tend to be purchased and set up by men (Strengers & Kennedy, 2020: pp. 42–47, pp. 178–185). Second, where oppressive gendered power relations play a large role in structuring relationships within a household, it is even less likely that women will be able to exercise authority over the settings of a smart speaker system. Third, relatedly, there is some evidence that where smart speakers are bought into such households they are often mobilised to extend the power of the man over the woman, which makes it unlikely that women will be able to adopt these systems to serve their own interests (Slupska & Tanczer, 2021). Indeed, to date most of the literature on gendered violence and smart speakers has been concerned with the use of these systems to spy on and control women (Bowles, 2018; Lopez-Neira et al., 2019; Maher et al., 2017; Mayhew & Jahankhani, 2020; Sovacool et al., 2021; Tanczer et al., 2021). At the very least, then, these considerations suggest that smart speaker systems will offer more to women in some domestic situations than others and, arguably, the least to those who might need them the most.

This ethical and political challenge does, however, have a practical/technological component relating to the nature of the settings on the device, the “default” settings, and how access to these is controlled. The approach manufacturers take to this practical challenge might offer some means to mitigate the concerns we have surveyed here.Footnote 10 For instance, installing “active assault detection” as a feature that could not be switched off would remove the onus from the woman to enable this feature. Allowing all members of the household to determine and alter settings via a mobile app, without these settings being visible to other users, might also empower women to turn on various features of a device that their partner would otherwise resist.

The practical challenge relates to the balance of the risks of false positives and false negatives when it comes to detecting (and taking action in response to) the target state of affairs. If a system generates too many false positives (that is, “detects” a target state of affairs when it is not actually present), this will undercut support for this technology, amongst women, and also, especially, amongst those who are expected to respond to alerts from the speakers. It is well known that people tend to develop “alert fatigue” when safety systems generate too many false positives (Agency for Healthcare Research and Quality, 2019). False positives will be especially problematic where they lead to police attending households given that the presence of the police may itself increase the risk of violence and associated harms to women (Dichter & Gelles, 2012). As we discuss further below, these risks are especially pronounced when members of households are also members of minority communities who have historically been disproportionately the victims of state violence and police harassment (Buxton-Namisnyk, 2022; Richie, 2012). Misidentification of victim-survivors as perpetrators, which historically has occurred at higher rates for First Nations women, culturally and linguistically diverse women, and women with disabilities than for other cohorts, is also a serious concern (Reeves, 2021). Conversely, systems that generate too many false negatives (that is, that fail to detect a target state of affairs when it is present) are likely also to undercut support for the product and to expose the manufacturers of smart speakers to legal and political risks. Inevitably—and unfortunately—measures to reduce the incidence of false positives tend to increase the risk of false negatives and vice versa. Getting the balance between rates of false positives and false negatives in smart speakers intended to identify risks to the safety of women in the home represents a profound challenge for the design of these systems, as it does in other contexts in which AI is being used in safety critical settings (Bland, 2020; Bowyer, 2004).

Social and ethical implications

It is a truism of science and technology studies that, by opening up new possibilities for action, the introduction of a new technology may also impact on social relations (Ihde, 1990; Vallor, 2016; Verbeek, 2011). In particular, new technologies often have implications for the sorts of things for which people might be held responsible, as well as for the distribution of responsibility for various outcomes (Jonas, 1973; Sparrow, 2023). More generally, new technologies often have social impacts that are unexpected (Winner, 1986). In this section we try to anticipate some of the consequences of developing and/or deploying smart speakers with the various capabilities described above, including their implications for social understandings of gendered violence. We argue that some of these consequences count as significant considerations against the wisdom of pursuing this technology.

One reason for reservations about the development of smart speakers to detect IPV is the extent to which it would represent a privatisation of policy responses towards IPV. Advocating the use of smart speakers to protect women from IPV risks implying that gendered violence is a problem in relationships between individuals that can be addressed in the home rather than a structural problem that reflects power relationships between the sexes in society more generally (Maher et al., 2017). At best this is naïve about the nature and causes of gendered violence: at worst, it represents a wilful refusal to confront the social and structural causes of such violence. Moreover, as should be obvious, detecting IPV, or a risk of such violence, is not a solution to IPV. Many survivors of IPV return to the relationships and the homes in which they were assaulted, for a range of reasons including genuine feelings for their partner, psychological dependence, or a lack of alternatives due to economic insecurity or other contingent circumstances (Heron et al., 2022). If a woman is in an abusive relationship, she is unsafe outside as well as inside the home. Reducing the risk of IPV requires, at a minimum, ensuring that women have realistic alternatives other than remaining with an abusive partner (Meyer, 2012). It almost certainly requires addressing the social structural causes of such violence. It may require providing resources to support those perpetrators, or potential perpetrators, of IPV who wish to address the social, historical, and psychological causes of their own behaviour in order to reduce the chance of their assaulting their partner (Forsdike et al., 2021). Placing smart speakers in the home does none of this. Worse, providing the opportunity to governments, and other social actors, to support the development or use of smart speakers to respond to IPV may reduce the political pressure on them to do what is actually required.

These reservations also have force in relation to the use of smart speakers to combat IPV insofar as it would rely on intervention by the police, or other agents of the state, to prevent IPV. There is a clear sense in which, if the police attend a home in response to a call about IPV, it is too late to prevent IPV. Moreover, women may believe there are negative consequences or “trade-offs” when seeking safety through police (Thomas, 2015) and, as reviews of police interventions and femicides show, police intercession does not automatically increase safety or prevent fatal violence (Independent Commission of Inquiry into the Queensland Police Service 2022). Police involvement (as well as involvement of other state agencies) in response to IPV may have different implications for members of different cultures. Members of some communities, including First Nations women, may view the police as dangerous and unwelcome due to their historical (and in many cases ongoing) role as enforcers of colonial governance (Buxton-Namisnyk, 2022; Gorrie, 2021). Interactions with social services (such as Child Protection) as a result of data gathered by smart speakers is likely to disproportionately impact First Nations women and women with disabilities, who already have their children removed at greater levels (George & Harris, 2014; Woodlock et al., 2014). Imprisonment of perpetrators as a result of police attending a household on the basis of an alert from a smart speaker may lead to deaths in custody, disruption of interfamilial relationships, and economic stress, which may harm victim-survivors and their families (Willis, 2011). It is worth noting, however, that these concerns also arise in relation to many other policy responses to IPV, which also rely on the police or the coercive power of the state more generally. Moreover, at least some of the potential uses of smart speakers to combat IPV—for instance, where they are deployed solely to warn women when the risk of IPV in a household rises above a certain level—probably escape these worries.

Another result of the development of smart speakers to detect IPV might be a change in expectations of women who are at risk of such violence and thus in the distribution of responsibility for the consequences of such violence (Maher et al., 2017: pp. 22 − 3). If the technology exists to detect IPV or the risk of IPV then an expectation may develop that women will use it (Gorfinkiel et al., 2021). If a woman does not make use of this technology and becomes a victim-survivor of IPV, the question may arise as to why she didn’t do more to protect herself (White & Lesley McMillan, 2020; Woodlock et al., 2020). Even if people are careful to insist—as they should be—that responsibility for violence rests with the perpetrator, just the perception that women have an option to reduce the risk of such violence by embracing surveillance at home may be enough to cause people to feel that victim-survivors of IPV are partially responsible for the consequences of IPV. That such an expectation is unreasonable, especially given the barriers to women accessing relevant functions on smart speaker systems discussed earlier, does not mean that it will not arise and play a role in shaping public attitudes towards victim-survivors of IPV.

Relatedly, relying on smart speakers to detect the various states of affairs surveyed above subtly—or perhaps not so subtly—disrespects and disempowers women by implying that their own experiences of gendered violence or inequality are not sufficient to allow the identification of these circumstances, to motivate action in response to them, or even for women to have an accurate perception of their own circumstances. Why not trust women themselves to identify and report these states of affairs? Why do we need a computer to do it (Morozov, 2013)? The use of smart speakers to alert the authorities to assaults in progress probably escapes this concern, as would the (hypothetical) detection of signals of risk of future assaults that are currently beyond the capacities of human beings to recognise. However, suggesting that we need smart speakers to identify threats, psychological violence, or gendered injustice neglects the availability of alternative policy responses to IPV that would involve listening to women themselves.

Insofar as smart speakers could only serve to detect any of the states of affairs surveyed above if they were “always on” and actively processing audio data continuously, advocating for the use of smart speakers to detect IPV is also to advocate that people should consent to pervasive audio surveillance by powerful corporations within their homes. This raises obvious issues relating to function creep (i.e. the use of such data by marketers, employers, and so on, and by police for other purposes) and privacy (Sovacool et al., 2021): the broader political consequences of such surveillance are yet to be reckoned with (Zuboff, 2019).

Finally, encouraging the uptake of smart speaker technologies by advertising their (putative) potential to protect women from IPV may actually have the effect of increasing the risks of emotional abuse and coercive control by making it easier for their partners to spy on, and harass, them. As we noted above, the use of smart speakers in this way by abusive partners is now well-documented.Footnote 11 It would clearly be a perverse outcome if the desire to protect women from IPV ended up facilitating such abuse.Footnote 12

To our minds, these potential social consequences of, and challenges associated with, the development of smart speakers to detect and respond to IPV establish a strong prima facie case against the wisdom of embracing Big Sister.

However, we are also conscious that others, who are more optimistic about the potential of this technology to prevent harm to women, may nevertheless think it is worth pursuing (Muraleedharan & Garcia-Constantino, 2022; Rodríguez-Rodríguez et al., 2019; Shah et al., 2018) especially given the evils of IPV. With the possible exception of the claim about the impact of the introduction of smart speakers that can detect IPV on the responsibilities of women, none of these social consequences are necessary consequences. It is possible, at least in theory, that a government, or a community, might endorse the use of smart speakers to detect incidents of IPV while also taking action to address the social/structural causes of IPV and to ensure that women in abusive relationships have access to resources to assist them in exiting violent relationships. Concerns about the impact of police attending minoritised households do not single out the use of smart speakers to combat IPV from existing approaches to responding to reports of IPV or indeed from law enforcement more generally. Governments might encourage women to consider installing such systems without blaming them in any way if they do not. They might insist that we should always listen to women, while also allowing that women might choose to install smart speaker systems that could report abuse in the event that they are unable to do so. Legislation might mitigate the risks of surveillance by restricting the uses to which the data gathered by “always on” smart speakers may be put. Finally, granted that a risk of abusive uses of smart speakers, both by individual men and by the corporations that manufacture them, exists, surely—it might be argued—it is better that these devices can also be used to detect and report incidents of IPV than not? Whether these options are realistically available—and thus how much the prima facie case against the development of smart speakers to detect IPV contributes to an all-things-considered assessment of this technology—turns on broader questions in STS about the political impacts of technology, which are, for the most part, beyond the scope of this essay.

Ethical decision points

There is, moreover, a further complexity that arises when considering the prospect of the development of smart speakers to detect and report IPV.

As we have seen, the attitudes towards IPV expressed by enthusiasm for smart speakers designed to detect and report IPV, as well as the likely social consequences of the adoption of such systems, count against the decision to develop the technology and/or to promote it once developed, at least to an extent. Given that part of the motivation for these sorts of objections is that there are alternative, better, responses to IPV, these concerns may have force even if there is some prospect that the technology, if it were to be developed, might work to protect individual women.

However, if this technology was actually developed, and shown to be capable of detecting incidents of IPV, the ethical landscape would look very different.

Once the technology becomes available, no individual decision to use it brings these consequences about or, indeed, makes them any more likely. Moreover, if deploying this technology would significantly reduce the risk of serious harms—including assault and murder—to women, the ethical case to do so is likely to be compelling. Importantly, this is true both at the level of the individual decision to use the technology and at the level of the decisions of manufacturers to include it in their smart speaker systems. Worries about the “expressive” content of the decision to deploy the technology—what this decision says about the person or institution deploying it and their ideas about what counts as an appropriate response to IPV—may still count against advertising it but seem unlikely to outweigh the good of reducing the risk of the harms associated with IPV in any particular case.

A similar logic suggests that were smart speakers to become capable of detecting incidents of IPV involving physical assaults in progress, the manufacturers of smart speakers would be morally obligated to ensure that the speaker would call the police. To do otherwise, would, in effect, be to stand by knowing that a woman was being assaulted by her partner and might be killed—which would be unconscionable. Although one would prefer that it did not matter enough to mention, failing to alert the police would also involve substantial political—and therefore economic—risk to the manufacturer. One can imagine the social media backlash if it became known that a smart speaker system had detected an assault, done nothing, and a woman had been killed. Thus, once it is demonstrated that smart speakers can detect assaults in progress, their use to do so, and to report such assaults to the police, is likely to be ethically and politically—and perhaps even eventually legally—mandated (Zenor, 2020: p. 860).

A further question arises about the threshold at which it would be obligatory to report the likelihood of physical violence as calculated by smart speakers or the extent of other harms to individual women associated with abuse short of physical violence. If such devices were capable either of accurately identifying households in which there was a high likelihood of an incident of physical violence occurring or of predicting imminent episodes of violence within a given household with a high degree of accuracy, then there would be a very strong case that this information should be reported to the police. There would also a strong ethical case for smart speakers to report the presence of other harms to women to relevant social services. Were such systems to become capable of identifying risks to women on the basis of signals in the data that were invisible to human beings, it seems that information about levels of risk should be communicated to the vulnerable party.

Importantly, these considerations generate at least three dynamics that are likely to pose real challenges for the manufacturers of such devices. First, especially when it comes to the risk of a serious assault, it will be difficult to defend not reporting—or at least taking some action—whenever the risk is higher than the background level of risk to (all) women. However, second, reporting the existence of a risk of IPV, even to the woman herself, but also to other social services and, especially, to the police is likely to be highly consequential: in some cases, it may itself impact negatively on the welfare of victim-survivors, and/or potential victim-survivors, of IPV, especially where they are First Nations persons or members of other minority communities (Buxton-Namisnyk, 2022). Getting the balance of considerations right here is likely to be highly ethically—not to mention legally and politically—fraught. Third, relatedly, reporting the risk of future violence, rather than the existence of an assault in progress, greatly exacerbates the problems associated with determining an appropriate balance between false positives and false negatives when it comes to the performance of the speaker system. Where action is taken to address a risk, it may be impossible to tell whether it was a false positive but—given that they would involve an assault—false negatives do not seem any less ethically (and politically) troubling (McCulloch & Wilson, 2015). Yet, as we observed above, systems that generate too many false positives are likely to lose the trust of users and fail for that reason. The combination and interaction of these dynamics may lead to the perverse outcome that manufacturers are obligated to install and maintain a technology that is highly likely to fail due to generating a high rate of false positives. For this reason, we suspect that critics of this technology may well find some support amongst the manufacturers of existing smart speakers.

Conclusion

Intimate partner violence is an urgent social and political problem, which has resisted solution via existing policy measures. “Smart” technologies for detecting IPV are already being developed: we suspect it is inevitable that more will be proposed in the years to come. As we have surveyed here, the widespread presence of smart speakers in domestic spaces offers an unprecedented opportunity, both rhetorical and real, to enlist “Big Sister” in the cause of combating IPV.

Such a prospect raises a raft of ethical and political challenges that we have highlighted in this paper and that call into question the wisdom of pursuing this project. In particular, Big Sister would privatise policy responses to IPV while failing to address its social and structural causes, and risk reducing the political pressure on governments to do the latter. It would raise, and perhaps intensify, worries about the implications of coercive solutions to IPV for minority communities. It would risk rendering women more responsible for their own safety while simultaneously disempowering them. It would involve intensive surveillance of domestic spaces and risk increasing the risks to women from abusive partners co-opting the devices that would conduct this surveillance.

These risks and concerns must be balanced against the moral urgency of the needs of those women—which is to say, arguably, all women—who are at risk of being abused, assaulted, or killed by their partners and who might benefit from Big Sister. Where we should judge this balance lies depends, in part, on larger questions about the extent to which technologies structure, and perhaps foreclose, political options, which we could not hope to resolve here. Detecting and reporting IPV, or risk factors for IPV, using smart speakers is not a solution to IPV: it needs to exist alongside initiatives that address the socio-economic structures that drive violence against women. Those who are more optimistic about human agency in relation to technology and its social consequences may believe that it is possible to pursue both these goals at once. The balance of considerations is also highly sensitive to the specificity of what smart speakers might be set to detect and what they might do upon detecting it.

Moreover, a key finding of our investigation is that the ethics of the decision to develop this technology and the ethics of the decision to use it once it exists differ significantly and thus the decision to develop this technology is highly ethically loaded. Concerns about what the embrace of Big Sister says about the origins and nature of IPV, the responsibilities of government and of women, and the appropriate limits of surveillance, have most force when we are considering the how we should respond to IPV and whether we should develop smart speakers to combat it. Once the technology has been developed, and is available, these concerns are mostly moot and the case to use it to protect individual women who are at risk of IPV is more compelling. Those who think that the embrace of Big Sister would be ethically and politically problematic would therefore be well-advised to make their criticisms early and to discourage the development of the technology rather than to try to resist its use once it arrives.

If this technology is developed, it will be important that those whose interests are supposed to be served by this technology are granted the opportunity to shape its final form. Engineers may have the technological know-how to extend the capacities of smart speakers but decisions about what to detect and what to do when a speaker detects it will, of necessity, involve assumptions about the nature, and causes, of IPV, as well as value judgements, for instance, about the relative importance of privacy and security. Those who are the victims-survivors of IPV have expert knowledge of these matters as well as a right to be heard on them. It is also clear that the use of smart speakers to combat IPV would need to be carefully regulated, especially given the extent to which it will draw on the resources of the police force and other state agencies. Again, then, both justice and a concern for good policy require that victim-survivors from a diverse range of backgrounds, as well as government and non-government stakeholders, be involved in formulating regulations of the use of smart speakers to combat IPV.

Finally, and for the same reasons, it will be essential that victim-survivors of IPV, as well as the community more generally, is included in the conversation, which we believe is urgently necessary, about the wisdom of embracing Big Sister. We hope our discussion here might play a useful role in prompting, and informing, this conversation.