Can Artificial Entities Assert? Forthcoming in The Oxford Handbook of Assertion, ed. Sanford Goldberg, Oxford University Press, 2020. Ori Freiman The Graduate Program in STS, Bar Ilan University, ori.freiman@biu.ac.il Boaz Miller Department of Community Information Systems, Zefat Academic College, boaz.m@zefat.ac.il Abstract There is an existing debate regarding the view that technological instruments, devices, or machines can assert or testify. A standard view in epistemology is that only humans can testify. However, the notion of quasi-testimony acknowledges that technological devices can assert or testify under some conditions, without denying that humans and machines are not the same. Indeed, there are four relevant differences between humans and instruments. First, unlike humans, machine assertion is not imaginative or playful. Second, machine assertion is prescripted and context restricted. As such, computers currently cannot easily switch contexts or make meaningful relevant assertions in contexts for which they were not programmed. Third, while both humans and computers make errors, they do so in different ways. Computers are very sensitive to small errors in input, which may cause them to make big errors in output. Moreover, automatic error control is based on finding irregularities in data without trying to establish whether they make sense. Fourth, testimony is produced by a human with moral worth, while quasi-testimony is not. Ultimately, the notion of quasi-testimony can serve as a bridge between different philosophical fields that deal with instruments and testimony as sources of knowledge, allowing them to converse and agree on a shared description of reality, while maintaining their distinct conceptions and ontological commitments about knowledge, humans, and nonhumans. Keywords technological instruments, epistemology, humans, quasi-testimony, technological devices, machine assertion, computers, testimony, knowledge, moral worth May Ashwortght, an eighty-six-year-old British woman, used to open her Google search queries with "please" and finish them with "thank you" because she assumed there was a person at Google's headquarters who looked after the searches (Ellis 2016). In 2018, Google introduced the Google Duplex chatbot, which makes it much harder to distinguish humans from computers. Duplex makes interactive phone calls to make restaurant reservations, schedule doctor appointments, and so on, in a human voice, natural language, and normal intonation.1 Duplex quickly faced criticism for deceiving people to think they are interacting with a real person, and for its potential to be misused for spamming and fraud. Google had not anticipated these problems (Griffin 2018). We regularly encounter automatic speech and text. We interact with personal assistant apps such as Apple Siri and Amazon Alexa. We hear automatic messages about train and flight departures and arrivals. We receive automatic calls from our credit card company about suspicious transactions and get statements about our bank account balance. We read news reports automatically generated by an algorithm.2 Some measuring instruments that used to have an analog interface, such as a dial or notches, now have digital verbal interfaces. Yet 1 For a demonstration, visit youtu.be/D5VN56jQMWM. 2 For an in-depth discussion of the issue of testimony and digital devices in the epistemology of journalism, see Godler et al. (forthcoming). 2 there is little philosophical discussion of these phenomena, and a standard view in wide philosophical circles is that only human persons can assert. This chapter critically surveys arguments for and against the view that technological instruments, devices, or machines can assert or testify. The existing debate uses the terms "assertion" and "testimony." According to some, like Fricker (1995, 396–397), testimony is synonymous with assertion: both terms describe the speech act of telling. According to others, such as Lackey (2008, ch. 1), testimony is a subclass of assertion. It is agreed by all sides, however, that if an entity can testify, it can assert. We therefore use both terms interchangeably. We introduce the notion of quasi-assertion (and quasi-testimony) and argue that it is attentive to considerations of different sides in this debate, and that it correctly characterizes assertion-like verbal outputs by technological devices. 1. The Problem of Testimonial Hybrids A debate whether nonhumans can testify has been conducted in the discipline of Science and Technology Studies (STS) within a larger debate about influential STS scholar Bruno Latour's views. This section applies Latour's criticism of modernity to analytic epistemology, to reveal its oversight of assertion by instruments. We discuss this oversight to introduce the issue of testimony and artificial entities, and then consider three approaches to remedy it. Latour (1993) suspects binary dichotomies associated with modernity, such as human/nonhuman and natural/artificial. He argues that an unbiased empirical examination of reality that does not assume such binary categories reveals that most things fit neatly into neither category. Such dichotomizing categories are not empirically given, but are secondorder abstractions made for classifying observed empirical reality. Because there can be more than one empirically valid classification system, a sociology of knowledge cannot explain the existence of a classification system by appealing to its truth. Rather, it must explain how this system emerged through interactions between the actants who constructed it. Specifically, it should explain how the abstract categories "natural" and "artificial" emerged in an empirical world that mostly consists of what Latour calls "hybrids," namely things that neatly fit into neither category. Latour's (1993) explanation is that dichotomizing categories are central to the very ideology of modernity, which either forces hybrids to uncomfortably fit in only one category or overlooks them. We may find similar tendencies in modern epistemology. Mainstream epistemology hardly addresses the ways perception is mediated and affected by technology, from simple eyeglasses to electron microscopes. Explicit analyses of knowledge from instruments are hardly present, and when one is found (e.g., Lehrer 1995), it analyzes knowledge from instruments in terms of inductive inference on perceptual beliefs, rather than as a distinct category of instrumental beliefs. In Latour's terms, modern epistemology represses the hybrid nature-both human and technological-of perceptual knowledge and treats it as a purely human phenomenon. Testimonial knowledge is similarly treated. A standard view in epistemology is that only humans can testify. Contemporary epistemology largely overlooks human/nonhuman3 testimonial hybrids, such automated announcements and computer voice interfaces. Human testimony is sharply contrasted with instruments. As Stephen Wright, for example, writes, testimony comes from speakers that have doxastic states that instruments lack. Speakers know things, where instruments do not [. . .] This supports the 3 Here, we use the notion of nonhumans in the sense of technologies, rather than nonhuman animals. 3 claim that there are certain attitudes that we can coherently take in response to testimony that we cannot coherently take in response to an instrument. (2014, 251; footnote omitted) Underlying this distinction is the view that instruments, like natural objects, exhibit law-like regularities, and therefore, like natural objects, are not subjectable to normative epistemic assessments. Elizabeth Fricker, for example, writes, we do not count thermometers, fuel gauges and so forth as testifying to the temperature, or to how much fuel is left in the tank. Nor do we count tree rings as testifying to the age of the tree. (2015, 178) Sanford Goldberg similarly claims: Information processing extends the process of belief-formation only when its operations are assessable in terms of the full range of normative assessments that go into epistemic assessment [. . .] This rules out information processing done by natural entities like trees; but it also rules out information processing done by mere instruments as well. (2012, 190) An opposite view, expressed by Ernest Sosa (2006), is that both humans and instruments exhibit law-like regularities. Sosa's view, however, simply dismisses the human qualities of testimony and instruments alike (we will say more about this in section 3). Explicit references in mainstream epistemology to testimonial hybrids are scares. The following somewhat dismissive excerpts by Fricker are an exception. Fricker discusses the distinctively human qualities of testimony and contrasts them with natural phenomena. She acknowledges a problem in her argument, which she brackets: (This contrast of natural and epistemic kind between natural versus agential meaning is muddied by the existence of what I think of as fake testimony: announcements at railway stations of train times, or automated messages one receives on telephone connections, that sound like a live human voice making statements, but are no such thing. I say a little about this phenomenon in my final section.) (Fricker 2015, 117) In the final section, she briefly returns to this issue: when a recipient hears, for instance, plane departure times being announced over a loudspeaker at an airport[, f]inding out that the utterances are produced by an automated artificial speaking mechanism does not, in this case, undermine the basis she has for believing what she hears. What matters is that the apparent speech act she perceives relays what is known; not that it is being produced by a speaker who herself knows what she states. (Fricker 2015, 201) Fricker's briefs remarks on testimonial hybrids ("fake testimonies") raise many questions. Are automated utterances genuine speech acts or just "apparent" ones? How possibly, in what sense, under what conditions, and by whom, if any, is the content of automated announcements known? How possibly, and by virtue of what, does a human recipient perceive whether an automatically asserted claim is known? Although the answers to these questions-which we review in this chapter-can severely undermine Fricker's sharp distinction between humans and inanimate nature, and despite the ubiquity of "fake testimonies," Fricker regards testimonial hybrids as an insignificant challenge, which can be 4 literally bracketed. By doing so, she vindicates Latour's point about the suppression of hybrids in modern philosophy. Whether one accepts Latour's claim that the suppression of hybrids is rooted in the very ideology of modernity, it is hard to deny its existence in analytic philosophy. When it comes to technologies, commonly accepted social epistemological approaches are insufficient because they mainly regard epistemic processes as socio-cognitive and neglect the epistemic roles of technologies in knowledge acquisition and belief justification. In the next sections, we examine three possible ways to address apparat assertions by instruments, beginning with denying a principled distinction between humans and nonhumans. 2. The Symmetry Principle: Denying a Distinction between Humans and Nonhumans Latour is critical of modernity for either forcing hybrids into binary categories or overlooking them. Latour's radical solution is the "symmetry principle": the denial of any a priori or principled distinction between human intentional action and a material world of causal relations (Latour 2005, 76). One motivation for Latour's symmetry principle is mainstream sociology's inattention to the causal and functional role of nonhumans. Latour (1992a, 1992b) argues that sociologists who pay attention to humans who perform certain actions overlook nonhumans who perform equivalent actions. Sociologists would take note of policepersons who slow down the traffic, but overlook speedbumps, which perform the same role. They would describe a human doorman as going on a strike, but not an automatic door closer that stops functioning. Specifically, Latour is critical of the Strong Program in the Sociology of Knowledge, which defines knowledge as a stable social agreement reached through testimonial negotiations by human subjects (Kusch 2002, ch. 5). Latour argues that the Strong Program leaves the natural world out of the explanation of knowledge. For a view to gain the status of knowledge, according to Latour, it must be negotiated and agreed upon by both human and nonhuman actants. In such negotiations and stabilization within a network of human and nonhuman actants, testimony of scientific laboratory objects has a special role. In a section entitled "The Testimony of Nonhumans," Latour (1993, 23) writes: [Laboratory objects are] inert bodies, incapable of will and bias but capable of showing, signing, writing, and scribbling on laboratory instruments before trustworthy witnesses. These nonhumans, lacking souls but endowed with meaning, are even more reliable than ordinary mortals, to whom will is attributed but who lack the capacity to indicate phenomena in a reliable way. Rather than explaining how scientists discover general laws, Latour explains how general laws are constructed in the laboratory as abstract idealizations, and diffuse outwards by negotiations between human and nonhuman actants to become an accepted description of reality. For example, Louis Pasteur developed in his laboratory the process of pasteurization, which prevents the development of microbes in food. But this process only works in a relatively sanitized environment like the laboratory. France of his time was far from such a sanitized environment. So to make his laws work outside the laboratory, Pasteur formed alliances with different actants to convince them to sanitize farms and hospitals (Latour 1988). Latour's symmetry principle has its share of problems. Latour is accused of unjustifiably anthropomorphizing inanimate objects (Amsterdamska 1990, 499; Brown 2001, 131; 5 Schaffer 1991, 182). In response, Latour argues that his anthropomorphism is justified because these hybrids are made by humans, substitute the actions of people, and shape human action. Moreover, he blames his modernist critics for having an anthropocentric bias: "You discriminate between the human and the inhuman. I do not hold this bias" (1992a, 236). Latour's critics argue that humans and nonhumans can be distinguished on principled ontological grounds (Bloor 1999; Collins 2010). Specifically, Harry Collins and Martin Kusch (1998) distinguish between actions that can be realized by different behaviors, such as paying (by handing cash, signing a check, etc.), and actions that cannot, such as swinging a golf club. They argue that machines can only perform the latter, whereas humans can perform both. As we saw, however, recent technological developments challenge this claim. Last, even Latour's sympathetic followers take his claims with a pinch of salt. Edwin Sayes-a sympathetic Latour interpreter-dismisses Latour's extravagant metaphysical statements as mere polemics, which "are neither viable claims nor, to be sure, consistent with the position's methodological comportment" (2014, 143). Sayes defends Latour as suggesting a deflationary account of nonhuman agency: The claim that nonhumans are full-blown actors can and should be understood only in a weak form-such a claim is demanded by both the methodological framework through which it emerges and the empirical world itself. (2014, 143) Latour's hyperbolic claim that artifacts have agency and can testify just like humans, then, boils down to claiming that artifacts can autonomously perform certain actions that are significant for explaining a certain phenomenon within a narrow explanatory context. Other than drawing attention to the often overlooked role of instruments in the generation of knowledge, it remains unclear what theoretical advantages are gained from Latour's insisting on indiscriminately calling instrumental output "testimonies." While methodologically fruitful, Latour's symmetry principle does not hold up to philosophical scrutiny. In the next section, we examine an alternative attempt to deal with human assertion as a natural regularity. 3. Human Testimony and Instruments as Natural Regularities Another way to argue that instruments can assert is denying that assertion possesses important distinctively human qualities. An argument to this effect is given by Sosa (2006). Sosa subscribes to externalism in philosophy. Roughly, externalists exclude from their philosophical analysis terms that refer to a person's subjective experience, namely, how the things are experiences to people "from within"; or they try to reduce such terms to terms that refer only to how things are objectively. Externalist epistemologists, Sosa included, analyze the conditions for obtaining knowledge in terms of counterfactual truth-tracking conditions a belief must meet, rather than the evidential support that the belief seems to have from the believing subject's perspective. Sosa argues that testimonial knowledge is a form of instrumental knowledge. First, both instruments and testimony exhibit law-like regularity in indicating the truth. Thus, the distinction between asserted and nonasserted outputs carries little epistemic importance. What matters is that both types of output reliably exhibit truth-tracking regularity on which 6 subjects rely to get knowledge (2006, 121). Second, language is itself a technology of communication for both speaker and hearer. Third, The man in the street [. . .] relies on his GPS devices, cellular telephones, atomic watches, and computer terminals with little or no awareness of how they depend on relations to other devices that more importantly seat the relevant functions. (2006, 117) In other words, according to Sosa, the "man in the street" relies on the de facto reliability of his instruments for gaining knowledge just like he relies on the reliability of testimony. He does not gain knowledge from an instrument by inferring that its output is true from background knowledge about its reliability, as an internalist account of knowledge (e.g., Lehrer 1995) would have it. Sosa's account, however, is misguided about both testimony and instruments. Sosa claims that we acquire knowledge from testimony because of its reliability: "Not easily would the speaker's utterance deliver that the speaker thinks (says) that such and such without the speaker's indeed thinking (saying) that such and such" (2006, 121). This claim seems empirically indefensible, and it leads to skepticism. People assert many things, some of them true and some false. We all encounter false and inaccurate testimonies. Pace Sosa, testimony is not generally reliable. If, as Sosa claims, we gain knowledge from testimony because testimony is generally reliable, then we don't get knowledge from testimony at all. Regarding instruments, at the heart of Sosa's account is a notion of a perfectly reliable, counterfactually sensitive, truth-tracking instrument. For Sosa, when we rely on instruments, "we make manifest our assumption of reliability" (2006, 118; emphasis in the origin). According to Sosa, we simply cannot establish the reliability of instruments using a coherence-involving method that fundamentally relies on perceptual beliefs (2006, 122). But a close examination of how instruments are designed, constructed, calibrated, and used reveals that Sosa's claim is false, and that his perfectly reliable instrument is a metaphysical fiction. Eran Tal's (2014) analysis of how standard time is produced by the International Bureau of Weights and Measures illustrates how reliability is established through achieving coherence between instruments. Standard time is produced by several highly reliable atomic clocks. They all realize an idealized definition of the standard second, as the duration of exactly 9,192,631,770 periods of the radiation corresponding to a hyperfine transition of caesium-133 in the ground state. As Tal notes, however, "in practice, no clock has a perfectly stable frequency. The very notion of a stable frequency is an idealized one, derived from the theoretical definition of the standard second" (2014, 301). This is because the standard second is defined with respect to conditions that cannot obtain in reality (e.g., being at rest at absolute zero degrees Kelvin). Every clock approximates this ideal definition differently and ticks at a slightly different pace. To minimize variance, the standard second is produced by a weighted average of the different clocks. Over time, if the clocks deviate from each other too much, they are recalibrated based on background theoretical assumptions about possible causes for their deviation. The weights assigned to the clocks are occasionally adjusted in a way that presumably achieves improved accuracy and prevents one clock from "hijacking" others over time. This method is generalizable to the way other standard units are de facto produced. If the most accurate atomic clocks in the world cannot achieve complete accuracy, all the more for the ordinary wristwatch. The standard second, then, is produced by calibrating several instruments that all realize the same idealized standard definition. This method replaces the old method of defining standards with respect to a concrete material object, such as a pole for the length of the standard meter, or a weight for the weight of the standard kilogram. But even these old concrete material standards were regarded neither as a completely accurate, nor as purely 7 conventional stipulative definitions. Rather, such objects are tentative, best available exemplars, and have to be constantly maintained to remain accurate. The standard meter, for example, had to be kept in constant temperature, and the standard kilogram had to be cleaned of small particles that somehow managed to penetrate the sealed tube in which it was. A similar analysis is applicable to the historical development of instruments. Hasok Chang (2004) philosophically analyzes the historical development of the thermometer. He describes how scientists overcame various problems they encountered in the process of its development. One such problem was discrepancy between different thermometers that worked on different principles, for example, water and mercury. While all thermometers within a given set would agree on zero and 100 degrees Celsius, they would show different temperatures when placed in the same lukewarm bath. Which one, if any, shows the correct temperature? Scientists overcame this problem by tweaking the thermometers to agree as much as possible based on background theoretical assumptions about possible sources for the discrepancy. They would then choose the thermometers they deemed best and use them as a new standard for developing the next generation of thermometers, and so forth. Chang calls this method epistemic iterations, and the corresponding theory of knowledge progressive coherentism, as it describes how improved accuracy is achieved over time by reaching agreement between instruments. He contrasts this with what he calls "foundationalism," according to which one instrument can serve as a fixed standard for all the rest (2004, ch. 5). A perfectly reliable instrument, or a perfect material standard, then, is a misguided metaphysical fiction, which does not accord with how even the most accurate instruments operate or develop over time. Even if a perfectly reliable instrument existed-for example, if there were an atomic clock that produced exactly the standard second-there would be no way to tell it apart from other highly accurate clocks. It would be analogous to a real barn in a fake-barn country, which does not generate knowledge in a subject that looks at it, even when she reliably forms the belief that it is a barn. While many instruments are reliable, their reliability is not brute. Like Leibniz, who assumes that once God set the universe in motion, it runs like a perfect machine, Sosa seems to assume that once an instrument is made, it is made reliable, and that this reliability, which is similar to the reliability of testimony, is why subjects gain knowledge from it. In reality, however, instruments are like the Newton-Clarke universe, in which God constantly intervenes and makes small adjustments. But it is not God who intervenes in this case, but human beings, who follow sui generis epistemic norms that pertain to the expected behavior of the interments. Instruments' reliability depends on their being designed, constructed, calibrated, continuously maintained, and correctly used. Knowledge from instruments depends not only on instrument makers but also on users who know and follow norms of use. In section 5, we argue that inasmuch as these norms are similar to the epistemic norms that govern assertion, a unified account of human and instrumental assertion may be developed. Before that, in the next section, we review considerations for regarding some instrumental outputs as assertions. 4. For and against Instrumental Testimony In the last section we discussed Sosa's attempt to treat testimony as a form of instrument which exhibits natural regularities, and we argued it mischaracterizes both instrumental and testimonial knowledge. Another possibility is to acknowledge testimony as a distinctive source of knowledge but argue that instruments can also testify. In this section, we review 8 the main arguments for and against this possibility, and we argue that testimony by instruments is a viable possibility. According to Jennifer Lackey, only persons testify. While Lackey speaks of persons in a broad sense, which includes nonhuman animals (2008, 189n13), instruments do not pass the threshold: "Persons, unlike other sources of belief, have all sorts of different intentions, desires, goals, motives, and so on" (2008, 189). At the same time, however, Lackey argues that a testifier need not believe what she testifies. Her example is a creationist teacher, who testifies the theory of evolution, while not believing it, hence not knowing it. But, as Billy Wheeler (2020) asks, if believing or having knowledge is not required for giving testimony, why can't instruments give testimony too? By waving on the demand that the source of testimony must know, and therefore have no intentions of passing what it knows, Lackey opens a theoretical possibility for an instrument to testify. In this spirit, Christopher Green (2010) argues that "machine testimony really is testimony." He claims that if two beliefs have the same epistemic status and content, are the result of the exercise of the same cognitive ability by the believing subject, and have the same phenomenology for the subject, then the two beliefs should be regarded as similarly based, that is, either both, or neither, as testimonial. He adds that characterizing beliefs from the linguistic output of machines using a category other than testimony multiplies epistemic categories beyond necessity.4 It might be objected that a human testifier has an intention to inform, regardless of believing what she informs, while an instrument does not. But as Wheeler (2020) argues, according to Lackey, there are also cases, for example, eavesdropping, in which a hearer acquires testimonial knowledge when the speaker does not intend to inform the recipient. An alternative reply to this objection, explored in Wheeler (2017), is that some instruments (robots) have the necessary intentions in the relevant sense or functional equivalences to such intentions. Goldberg objects to this line of reasoning. As we saw in section 1, Goldberg maintains that human testimony is subject to normative epistemic assessment, but instrumental outputs are not. Since whether a belief is justified is a normative assessment, a human testifier's normative epistemic conduct can affect the justificatory status of a belief the recipient of her testimony forms from it. But the behavior of an instrument on which a subject relies to form a belief does not have this affect. An instrument can cause beliefs, but not affect their justificatory status. The justificatory status of a belief a subject acquires from an instrument is entirely due to her own epistemic conduct: insofar as you are relying on the thermometer's output, your reliance is only as epistemically good as your own grounds for so relying (observed track record; reasons for trusting the company that manufactured it; and so forth). [. . . W]hereas reliance on another epistemic subject can make you epistemically dependent on that subject, reliance on a mere instrument (or other item in your environment) does not do so. (Goldberg 2013, 29, emphasis in the origin) Against Goldberg, Wheeler (2020) describes Jim, a professional burglar, and Jill, a trader who buys a stolen diamond from him "asking no questions." While both Jim and Jill violate moral and legal norms, they violate different norms. Jim is guilty of stealing, while Jill is guilty of 4 A similar criticism that builds upon Goldberg's asymmetrical conceptual boundaries between humans and nonhuman agents in regard to testimony is made by Paul Smart (2017). 9 trading in stolen goods. The normative assessment of Jim does not transfer to Jill (we can even think of a case in which Jill buys a stolen diamond by being misled to think it is legitimate, thus violating no norms). The diamond is analogous to a true belief, where Jim is a testifier and Jill is its recipient. The upshot is that while there may be epistemic norms of assertion, which apply to testifiers, they are not the same as norms of belief. Only norms of beliefs-which apply only to the recipient of testimony, are relevant to justificatory status of the beliefs. Hence, the fact that instruments are not subjectable to normative assessment does not preclude the possibility of their offering testimony, or so Wheeler argues. An alternative reply to Goldberg is that while instruments as such are not subjectable to normative epistemic assessment, instrument makers and maintainers are, similarly to human asserters. Thus, there is no basis for drawing a principled distinction between humans and instruments with respect to their ability to offer testimony. Indeed, in a later paper, Goldberg (2017), partly retracts from his previous position, which equates instruments with natural regularities such as tree rings, and argues that the norms that guide instrument makers foster among instrument users epistemically legitimate expectations, which are indirectly relevant to the assessment of the justificatory status of the beliefs their users form from these instruments. Still, so Goldberg (2017, 2–3) insists, even if instrument makers' epistemic norms are relevant to the justificatory status of instrument users' beliefs, there is a principled difference between instruments and testimony as sources of knowledge.5 Testimony is the result of cognitive processes in a human testifier's cognition. Only when these cognitive processes are reliable, do they confer justification on a belief that a recipient of the testimony forms from it. By contrast, the processes in an instrument, which produce its output, are merely causal and therefore cannot confer justification on the respective beliefs of its user. Goldberg's distinction, however, is unsatisfactory. Goldberg's claim that only cognitive (which for Goldberg means psychological) processes can confer justification on a belief seems question begging, since at the heart of the debate is exactly whether nonpsychological creatures can offer testimony. Moreover, Goldberg's claim that information-processing processes in instruments are not cognitive challenges Clark and Chalmers' plausible (though not uncontroversial) parity principle, according to which: If, as we confront some task, a part of the world functions as a process which, were it done in the head, we would have no hesitation in recognizing as part of the cognitive process, then that part of the world is [...] part of the cognitive process. (1998, 8; emphasis in the origin) One might also argue-against Goldberg-that the cognitive processes that occur in instrument makers' cognition can directly confer justification on instrument users' beliefs. Only when these processes lead instrument makers to make reliable instruments, including instruments that give reliable testimonies, is a belief acquired from them justified. This is similar to how, according to Goldberg, reliable cognitive processes in a testifier's cognition confer justification on the beliefs formed by the testimony recipient. Against this, Goldberg may argue that instrument makers' cognitive processes do not generate a particular output, whereas a human testifier's cognitive processes do.6 It follows that in the case of instruments, but not in the case of testimony, there is no subject that is responsible or can give her assurance for the content of a particular belief. For example, when 5 Neges (2018, 179–214) argues that some knowledge from instruments can be analyzed as a novel epistemic source, not reducible to other basic epistemic sources-testimony included. 6 We thank Sandy Goldberg for this objection. 10 Sam checks his daughter for fever with a thermometer, the thermometer makers may have given their assurance for its general reliability, but not for the specific reading that Sam's daughter's fever is 38.2°C, for which, according to Goldberg, only Sam is epistemically responsible. By contrast, when Sam asks Penny if the clinic is open, and she says that it is, she gives him her personal assurance for that claim, and she is epistemically responsible for it. This objection, however, fails to demarcate instruments and testimony as sources of knowledge. There are cases of testimony in which the testifier's cognition does not generate the particular asserted claim, and the testifier is responsible only for the accuracy of transmission of a message. Consider Constable Azoulay, who is a patrol officer. He walks the streets, and wonders how his wife Betty is doing. He asks the paperboy, who runs and asks a woman hanging the laundry in her balcony, who shouts to Betty's neighbor, who shouts to Betty, who is in her apartment. "I'm fine," Betty shouts back to her neighbor. "She's fine," her neighbor shouts to the laundry woman. "She's fine," the laundry women shouts to the paperboy. The paperboy runs back to Constable Azoulay and says, "She's fine, Constable."7 All the participants in the testimonial chain other than Betty are neither responsible nor give their assurance for the content of the message, only for its accurate transmission. They function as mere information transmission instruments. It might be objected that human testifiers still have a responsibility to monitor the source and plausibility of the message they pass on. But it is unclear that human testifiers always have this responsibility. If Constable Azoulzy challenges the paperboy and asks, "How can Betty be fine? She was sick in bed when I left home this morning," the paperboy may legitimately reply, "I don't know, sir. I'm only passing on what I've been told." Last, Goldberg's claim that instruments are not subjectable to normative epistemic assessment is only partly correct. While we do not ascribe epistemic praise or blame to mere instruments, we do have expectations from them, which are normative, unlike our expectation from natural objects, such as the sun and moon. Additionally, instruments are subjectable to normative epistemic assessment in that we can spell out desiderata for their expected behavior. For example, Miller and Record (2017; cf. Miller and Record 2013) analyze Internet search autocomplete as a form of testimony and derive principles for epistemically responsible autocomplete. They argue that in light of their desiderata, epistemically responsible autocomplete cannot be fully automated, and there must be human judgment involved in the process. Miller and Record come to this conclusion, however, only after they specify the desiderata. When the desiderata for epistemic responsibility are different or the technology is different, instruments may satisfy them, and in that sense comply with our normative epistemic expectations. So far, we reviewed arguments from a philosophical externalist perspective against the possibility of instrumental testimony, and we found them wanting. An argument that some instruments can give testimony can be made on internalist grounds as well, drawing on similarities in subjects' experience of verbally interacting with humans and some technologies. Such an argument is made by Galit Wellner (2014), who phenomenologically analyzes the smartphone and argues that it constitutes a "quasi-face." Drawing on Don Ihde (2009), who identifies four possible types of human-technologyworld relations, Wellner characterizes user-smartphone-world relations as instantiating two relation types: hermeneutical and alterity. In hermeneutical relations, subjects interpret technological artifacts using what Ihde (2009, ch. 4) calls "material hermeneutics." According to Ihde, "things speak" and humans interpret their talk. Material hermeneutics applies to both natural objects and technological instruments. Ihde illustrates this notion with Ötzi, a three- 7 This example is inspired by the Movie The Policeman (1971), directed by Ephraim Kishon. 11 thousand -year-old iceman that was found in the Austrian Alps in 1991. In order to determine its age, scientists had to interpret readouts from various instruments that mediate empirical inputs into visual outputs. Another form of relations that exist between humans and smartphones, according to Wellner, is alterity relations, in which a person experiences technology as a distinct other. Examples for alterity relations are interacting with a personal assistant such as Siri, playing chess with a computer, or using an ATM, with which one interacts "as if it were a human teller in a bank. But it is not, it is merely a computerized machine with a screen. It provides communication with a quasi-other" (Wellner 2014, 312). Wellenr's argument that alterity relations exist between users and their smartphone to the extent the smartphone screen can be characterized as a quasi-face that gives testimony draws on two elements from Levinas's philosophy. The first is Levinas's (1995) claim that the house is a fundamental technology to human life, as it enables the separation between the private and public sphere. The house shields and hides its residents from the outside world, thus physically creating the private sphere, but it also allows interaction through its windows and doors. The second is Levinas's analysis of the human face as the interface organ between the self and other. As the central organ that is used for verbal and nonverbal communication, the face is the main vehicle that allows us to recognize a thinking, conscious mind located in the body of another person. Wellner argues that a smartphone screen bears similarity both to the house and the face. Like the house, the screen is a material technology. It is shaped as a window, and like a house, it opens virtual windows in it. Its similarity to the face is fourfold: First, both the face and the screen represent interiority without it being externalized. Like a foyer of a home which does not reveal the inside, the screen is the exteriority of software codes [. . .] Second, the screen, like the face, serves as a communicative interface. While the face serves as an inter-face point between humans, the screen serves as an interface point between humans and their technologies [. . . ] Third, both types of interface function on the basis of language. [. . .] The interaction with the screen amounts to a language in the sense of providing commands and performing certain actions [. . .] Fourth, the face and the screen extend beyond ''physical boundaries.'' The face's extensions are other expressive organs such as the hands. The screen's extensions are peripherals like the keypad of the cell phone or the microphone. (Wellner 2014, 312–313) Wellner recognizes a significant difference, however, between quasi-face and genuine human face. A human face demands an ethical response, because it is recognized as encasing another person's conscious mind worthy of being treated with dignity. A quasi-face does not necessarily demand such an ethical response. Generalizing from Wellner's analysis of the smartphone screen as a quasi-face, we suggest that at least in some cases in which the relations between a human user and a technology are both hermeneutical relations and alterity relations, the technology can be characterized as making an assertion or testimony. 5. Insincerity and Quasi-Testimony In the last section we argued that instrumental testimony is a viable possibility, but we did not spell out conditions under which an instrumental output constitutes testimony. We now 12 introduce the notion of "quasi-testimony," which addresses this question. We develop it from Philip Nickel's and Wheeler's idea that a machine can assert only when it is capable of insincerity. Nickel (2013) and Wheeler (2020) draw on the following observation by Bernard Williams: In the case of this machine there is a direct route from the state that it is in to what it prints out; or if something goes wrong on this route, it goes mechanically wrong, that is, if something interrupts the connection between the normal inner state for asserting that p and its asserting that p, and it comes out with something else, this is merely a case of breakdown. It is not a case of insincere assertion, of its trying to get you to believe that p when really all the time it believes itself that not-p: we have not given it any way of doing that. [ . . . W]hen I said this machine made assertions, I should have actually put that in heavy scare-quotes; "assertion" itself has got to be understood in an impoverished sense here, because our very concept of assertion is tied to the notion of deciding to say something which does or does not mirror what you believe. (1973, 145–146) For a machine to be a proper agent, Williams argues that it needs to have belief-like representations of the world and the ability to make inferences based on them. As we saw, however, Wheeler denies that a speech actor needs to have beliefs. Wheeler's rationale for the insincerity requirement is that one's having the ability to deceive and withhold information is essential for one's being a functioning member of an epistemic community, for example, in a scientific priority race. Moreover, Wheeler argues, one's having the ability to deceive and others' being aware of one's ability are required for others to develop and exhibit an attitude of trust toward one's putative assertions, as opposed to mere reliance. Wheeler discusses experiments in which subjects played games with robots. When the robots were deceptive, subjects treated them as human players. Nickel and Wheeler disagree with Williams that a machine is incapable of insincerity. As Nickel argues: When the function of a machine is to report the truth, it is possible for its utterances to count as insincere in the event that it deviates from the norm of truthfulness for non-mechanical reasons. This is parasitic on ordinary norms of assertion, which has as a speech act the function of representing the truth [. . . ] When a designer is aware that the machine has this function, and that the corresponding norm of assertion applies, but intentionally programs it to not to meet the norm for some ulterior end, the machine creates insincere speech. (2013, 492–493; emphasis on the origin) Machine insincerity is not a merely theoretical possibility. When you call a call center and hear an automatic message that "your call is third in line," this is not necessarily true. The asserted place in the queue may not be the real place, but a number the system designers think will make you wait patiently without hanging up.8 Car manufacturers design the speedometer to display a value higher than the actual speed of the car for safety and liability reasons (Will 2010). Speaking sex dolls are designed to realistically fake pleasure (Williams 8 We learned this in private communication with an engineer in a company that develops such systems. 13 2019 ). Health robots should arguably sometimes deceive patients about their real condition (Matthias 2015). The insincerity requirement, however, is too strong as a necessary condition for machine assertion. As Nickel (2013, 494) notes, it leaves out mere measurements of physical magnitudes, as well as simple, scripted, context-restricted automated messages, such as flight and gate announcements on an airport loudspeaker. We find this exclusion problematic. If a verbal announcement on an airport loudspeaker constitutes an assertion when it is made by a human employee, why doesn't the same verbal announcement on the same loudspeaker constitute an assertion when made by a computer? The function of the message, the explanation of why subjects get knowledge from it, and the phenomenology are the same in both cases. That an employee can be insincere and a computer cannot does not constitute a good reason to distinguish the two in this context.9 We therefore propose a less restrictive criterion for quasi-testimony (or quasi-assertion) by a machine or an instrument. Quasi-Testimony A linguistic output of an instrument or a machine constitutes a quasi-testimony in a given context of use if and only if the machine or instrument has been designed and constructed to produce this output in a manner that sufficiently resembles testimony phenomenologically, and it is in conformity with an epistemic norm that is parasitic on, or sufficiently similar to what is, or would be, an epistemic norm of testimony in the same context. The prefix "quasi" indicates that while instruments assert or testify, they do so differently from humans. The idea is that an instrumental output constitutes a quasi-testimony when, by design, it sufficiently resembles human testimony.10 When this happens, the instrument designer counts on its users to correctly decipher the meaning of the output and correctly assess its validity because they recognize the testimony-like epistemic norms under which the output is produced. A straightforward way to make users recognize these norms is making the output phenomenologically similar to human testimony. An automated announcement in natural language on a loud speaker is a typical example. It uses the same medium (language) and phrasing a human would use, and this is why people understand it and its function so easily. A digital timestamp on a photograph is a more complex example. By itself, it is merely a printed series of numbers. But we immediately recognize it as referring to the time the photograph was taken (being aware that it can be false as well). It functions like a person testifying: "This photograph was taken on this date at this time." Our quasi-testimony condition captures the two rationales behind Wheeler's and Nickel's insincerity condition. It holds the output to the relevant epistemics norms, and it allows subjects to develop an attitude of quasi-trust toward the source. Quasi-trust is distinguishable from mere reliance in that it is grounded in normative expectations from its 9 Our claim here echoes Green's and Fricker's claims, reviewed in sections 4 and 1. Nickel formulates the insincerity condition as follows: "The message could in principle be insincere, in the sense that it deviated intentionally ('by design') from relevant norms of assertion" (2013, 493). It seems that for Nickel, "in principle" means that a machine actually has the capacity for insincerity even if it does not manifest it. But if "in principle" means that a machine can be designed to produce false messages, even if it is not actually designed this way, we do not object to this condition. 10 For a proposal in a similar spirit to treat as legal testimony some instrumental outputs that are presented as evidence in court, see Roth (2017). 14 target , as opposed to expectations that are based merely on inductive inference about its reliability. Not all instrumental outputs constitute quasi-testimony. A photograph, an MRI imaging, an ultrasound imaging, and an ECG graph are not quasi-testimonies-as they are not linguistic outputs.11 A measurement that needs to be interpreted also does not constitute quasi-testimony. However, when measurement procedures are black-boxed and reading off the outcome is straightforward, it may constitute quasi-testimony. Anyway, a determination of whether an instrumental output constitutes quasi-testimony requires an empirical investigation into the phenomenology of the output, the epistemic norms of the use of the instrument, and "practices of trust" (Record and Miller 2018, 110–117) users perform to establish their epistemic dependence. Why regard this class of instrumental outputs as quasi-testimony, rather than genuine testimony, and the corresponding attitude as quasi-trust, rather than genuine trust? We reviewed two sides of a debate. On the one hand, Latour and Sosa deny a distinction between humans and instruments with respect to the possibility to assert. On the other hand, Goldberg claims there are fundamental differences. Like Wellner, we argued that both positions are wrong: there are relevant differences between humans and instruments that militate against treating them exactly alike; but these differences are not enough for endowing only humans with assertion. The relevant differences between humans and instruments are fourfold. First, unlike humans, machine assertion is not imaginative or playful. A flight attendant may say. "Anybody found smoking in the lavatory during flight will be taken off the aircraft immediately." Human passengers will understand this is a joke, but machines cannot generate or understand (yet) such playful assertions. Second, machine assertion is prescripted and context restricted. Unlike humans, computers currently cannot easily switch contexts or make meaningful relevant assertions in contexts for which they were not programmed. Current best attempts to pass a Turing test still use circumventing strategies such as trying to change the subject or feign misunderstanding when the computer encounters an unintelligible context. While Google Duplex, which was mentioned in the introduction, can skillfully handle scripted scenarios such as making a restaurant reservation, it cannot-in all likelihood-skillfully answer an out-of-context question or generate an associative joke. Third, while both humans and computers make errors, they err in different ways. For example, a person was automatically charged US$81 billion on his credit card for filling a gas tank (Trumbo 2009). No human would mistakenly charge this amount for a gas tank or bother to calculate it to the very last cent. This illustrates that unlike humans, computers are very sensitive to small errors in input, which may cause them to make big errors in output. Whereas a human would see that such an amount doesn't make sense,12 or that a comma is missing between two numbers, a computer would not, unless explicitly programmed to do so. Moreover, automatic error control is based on finding irregularities in data without trying to establish whether they make sense. For example, Luke has never traveled abroad. Planning 11 For an opposite view, according to which a visual image can also constitute testimony, see Schankweiler et al. (2019). 12 Or so we hope. The person mistakenly charged had to argue with a human representative for about fifteen minutes to have the charge corrected. 15 his first trip, he calls his credit card company to open his card for use abroad. An algorithm identifies this as an irregularity and blocks the card for use abroad.13 Fourth, as noted, testimony is produced by a human with moral worth, while quasi-testimony is not. Testimony is a central notion in postmodern ethics, which struggles with the question of how to make right moral decisions when there are supposedly no moral truths. A proposed solution is making the demand to respond to the testimony of other human beings, especially victims, a primary moral principle. As Michal Givoni (2016) argues, this solution is naive, because testimony can be manipulative and misleading. But in this context, the difference between testimony and quasi-testimony is important. These differences are not a priori but refer to the current state of technology. As long as the operating principles of quasi-testimony-generating machines do not change, namely, as long as they are digital, code-running machines, these differences are here to stay, even if we witness great advances in processors' speed and storage capacity (Kuflik 1999; Miller and Record 2017, 1954–1955). But history may prove us wrong, like it proved wrong Descartes, who thought that a machine that made meaningful verbal responses to verbal stimuli was inconceivable (Descartes 1985, 140; quoted in Nickel 2013, 490). If in the future machines can make unscripted, playful assertions, transcend context, intelligently monitor errors, or develop consciousness that gives them moral worth, we will need to rethink the notion of quasi-testimony. 6. Conclusion We introduced quasi-testimony as a notion that acknowledges that technological devices can assert or testify under some conditions, without denying that humans and machines are not the same. Our suggested notion can serve as a bridge between different philosophical fields that deal with instruments and testimony as sources of knowledge, particularly analytic philosophy of language, social epistemology, STS, postphenomenology, and postmodern ethics.14 The notion of quasi-testimony can serve as a "boundary object" (Star and Griesemer 1989) or a "trading-zone" concept (Galison 1997, ch. 9) between these scholarly traditions, allowing them to converse and agree on a shared description of reality, while maintaining 13 This example is based on a true story. 14 Previous attempts to use testimony as a bridge concept between different philosophical traditions include Lipton (1998), Fricker (1998), Kusch (2002), and Solomon (2007). They were only partly successful because they did not attend to knowledge from technological instruments. 16 their distinct conceptions and ontological commitments about knowledge, humans, and nonhumans. References Amsterdamska, Olga. 1990. "Surely You Are Joking, Monsieur Latour!" Science, Technology & Human Values 15, no. 4: 495–504. Bloor, David. 1999. "Anti Latour." Studies in History and Philosophy of Science 30, no. 1: 81– 112. Brown, James R. 2001. Who Rules in Science? An Opinionated Introduction to the Wars. Cambridge, MA: Harvard University Press. Chang, Hasok. 2004. Inventing Temperature: Measurement and Scientific Progress. Oxford: Oxford University Press. Clark, Andy, and David Chalmers. 1998. "The Extended Mind." Analysis 58, no. 1: 7–19. Collins, Harry M. 2010. "Humans Not Instruments." Spontaneous Generations 4, no. 1: 138– 147. Descartes, Rene. 1985. "Discourse on the Method." In The Philosophical Works of Descartes, vol. I. Translated by J. Cottingham, R. Stoothoff, and D. Murdoch. Cambridge: Cambridge University Press. Ellis, Catherine. 2016. "Grandson Shares Nan's Polite Google Search." BBC News (June 15). http://www.bbc.com/news/uk-36538356 Fricker, Elizabeth. 1995. "Telling and Trusting: Reductionism and Anti-Reductionism in the Epistemology of Testimony." Mind 101, no. 414: 393–411. Fricker, Elizabeth. 2015. "How to Make Invidious Distinctions amongst Reliable Testifiers." Episteme 12, no. 2: 173–202. Fricker, Miranda. 1998. "Rational Authority and Social Power: Towards a Truly Social Epistemology." Proceedings of the Aristotelian Society 98, no. 2: 159–177. Galison, Peter. 1997. Image and Logic: A Material Culture of Microphysics. Chicago: University of Chicago Press. Givoni, Michal. 2016. The Care of the Witness: A Contemporary History of Testimony in Crises. Cambridge: Cambridge University Press. Godler, Yigal, Zvi Reich, and Boaz Miller. Forthcoming. "Social Epistemology as a New Paradigm for Journalism and Media Studies." New Media and Society. Goldberg, Sanford C. 2012. "Epistemic Extendedness, Testimony, and the Epistemology of Instrument-Based Belief." Philosophical Explorations 15, no. 2: 181–197. Goldberg, Sanford C. 2013. "'Analytic Social Epistemology' and the Epistemic Significance of Other Minds." Social Epistemology Review and Reply Collective 2, no. 8: 26–48. Goldberg, Sanford C. 2017. "Epistemically Engineered Environments." Synthese. https://link.springer.com/article/10.1007%2Fs11229-017-1413-0 Green, Christopher R. 2010. "The Epistemology of Testimony." The Internet Encyclopedia of Philosophy. http://www.iep.utm.edu/ep-testi/ Griffin, Andrew. 2018. "Google Duplex: Why People Are So Terrified by New HumanSounding Robot Assistant." The Independent (May 11). 17 http ://www.independent.co.uk/life-style/gadgets-and-tech/news/google-duplex-whyexplained-controversy-objections-ai-artificial-intelligence-robot-a8347566.html Ihde, Don. 2009. Postphenomenology and Technoscience: The Peking University Lectures. Albany, NY: SUNY Press. Kuflik, Arthur. 1999. "Computers in Control: Rational Transfer of Authority or Irresponsible Abdication of Autonomy?" Ethics and Information Technology 1, no. 3: 173–184. Kusch, Martin. 2002. Knowledge by Agreement: The Programme of Communitarian Epistemology. Oxford: Oxford University Press. Lackey, Jennifer. 2008. Learning from Words. Oxford: Oxford University Press. Latour, Bruno. 1988. The Pasteurization of France. Cambridge, MA: Harvard University Press. Latour, Bruno. 1992a. "Where Are the Missing Masses? The Sociology of a Few Mundane Artifacts." In Shaping Technology/Building Society: Studies in Sociotechnical Change, edited by Wiebe E. Bijker and John Law, 225–258. Cambridge, MA: MIT Press. Latour, Bruno. 1992b. "One More Turn after the Social Turn." In The Social Dimensions of Science, edited by Ernan McMullin, 272–294. Notre Dame: Indiana University Press. Latour, Bruno. 1993. We Have Never Been Modern. Cambridge, MA: Harvard University Press. Latour, Bruno2005 .. Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford: Oxford University Press. Lehrer, Keith. 1995. "Knowledge and the Trustworthiness of Instruments." The Monist 78, no. 2: 156–170. Levinas, Emmanuel. 1995. Totality and Infinity: An Essay on Exteriority. Pittsburgh: Duquesne University Press. Lipton, Peter. 1998. The Epistemology of Testimony. Studies in History and Philosophy of Science 29, no. 1: 1–31. Matthias, Andreas. 2015. "Robot Lies in Health Care: When Is Deception Morally Permissible?" Kennedy Institute of Ethics Journal 25, no. 2: 169–162. Miller, Boaz, and Isaac Record. 2013. "Justified Belief in a Digital Age: On the Epistemic Implications of Secret Internet Technologies." Episteme 10, no. 2: 117–134. Miller, Boaz, and Isaac Record. 2017. "Responsible Epistemic Technologies: A SocialEpistemological Analysis of Autocompleted Web Search." New Media & Society 19, no. 12: 1945–1963. Neges, Sebastian. 2018. Instrumentation. A Study in Social Epistemology. PhD diss., University of Vienna. Nickel, Philip. 2013. "Artificial Speech and Its Authors." Minds and Machines 23, no. 4: 489– 502. Record, Isaac, and Boaz Miller. 2018. "Taking iPhone Seriously: Epistemic Technologies and the Extended Mind." In Extended Epistemology, edited by Duncan Pritchard, Andy Clark, 18 Jesper Kallestrup, Orestis Palermos, and J. Adam Carter, 105–226. Oxford: Oxford University Press. Roth, Andrea. 2017. "Machine Testimony." The Yale Law Journal 126, no. 7: 1972–2053. Sayes, Edwin. 2014. "Actor–Network Theory and Methodology: Just What Does It Mean to Say that Nonhumans Have Agency?" Social Studies of Science 44, no. 1: 134–149. Schaffer, Simon. 1991. "The Eighteenth Brumaire of Bruno Latour." Studies in History and Philosophy of Science 22, no. 1: 174–192. Schankweiler, Kerstin, Verena Straub, and Tobias Wendl. 2019. "Image Testimonies: Witnessing in Times of Social Media." In Image Testimonies: Witnessing in Times of Social Media, edited by Kerstin Schankweiler, Verena Straub, and Tobias Wendl, 1–13. Abingdon: Routledge. Smart, Paul R. 2017. "Extended Cognition and the Internet: A Review of Current Issues and Controversies." Philosophy & Technology 30, no. 3: 357–390. Solomon, Miriam. 2007. STS and Social Epistemology of Science. In The Handbook of Science and Technology Studies, 3rd ed., edited by E. J. Hackett, O. Amsterdamska, M. Lynch, & J. Wajcman, 241–258. Cambridge, MA: MIT Press. Sosa, Ernest. 2006. Knowledge: Instrumental and Testimonial. In J. Lackey & E. Sosa (Eds.), The Epistemology of Testimony (pp. 116–123). Oxford: Oxford University Press. Star, Susan Leigh, and James R. Griesemer. 1989. "Institutional Ecology, Translations and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology," 1907–39. Social Studies of Science 19, no. 3: 387–420. Tal, Eran. 2014. "Making Time: A Study in the Epistemology of Measurement." The British Journal for the Philosophy of Science 67, no. 1: 297–335. Trumbo, John. 2009. "Driver Fills Tank in Richland, Drains Bank." Tri-City Herald (February 26). http://web.archive.org/web/20110813043134/http://www.tricityherald.com/2009/02/26/491472/driver-fills-tank-in-richland.html Wellner, Galit. 2014. "The Quasi-Face of the Cell Phone: Rethinking Alterity and Screens." Human Studies 37, no. 3: 299–316. Wheeler, Billy. 2017. "Giving Robots a Voice: Testimony, Intentionality, and the Law." In Androids, Cyborgs, and Robots in Contemporary Culture and Society, edited by Steven John Thompson, 1–34. Hershey, PA: IGI Global. Wheeler, Billy. 2020. "Reliabilism and the Testimony of Robots." Techné 24, no. 2. Will, Joanne. 2010. What's More Accurate: The Car's Speedo or the GPS? The Globe and Mail (November 17). http://www.theglobeandmail.com/globe-drive/culture/commuting /whats-more-accurate-the-cars-speedo-or-the-gps/article4348596/ Williams, Alex. 2019. "Do You Take This Robot..." The New York Times (January 19). http://www.nytimes.com/2019/01/19/style/sex-robots.html Williams, Bernard. 1973. Problems of the Self. Cambridge: Cambridge University Press. Wright, Stephen. 2014. "Sosa on Knowledge from Testimony." Analysis 74, no. 2: 249–254.