Response to Floridi et al, 2008/2009. Based on insufficient evidence, and inadequate research, Floridi and his students report inaccuracies and draw false conclusions in their Minds and Machines evaluation, which this paper aims to clarify. Acting as invited judges, Floridi et al. participated in nine, of the ninety-six, Turing tests staged in the finals of the 18th Loebner Prize for Artificial Intelligence in October 2008. From the transcripts it appears that they used power over solidarity as an interrogation (...) technique. As a result, they were fooled on several occasions into believing that a machine was a human and that a human was a machine. Worse still, they did not realise their mistake. This resulted in a combined correct identification rate of less than 56%. In their paper they assumed that they had made correct identifications when they in fact had been incorrect. (shrink)

A series of imitation games involving 3-participant (simultaneous comparison of two hidden entities) and 2-participant (direct interrogation of a hidden entity) were conducted at Bletchley Park on the 100th anniversary of Alan Turing’s birth: 23 June 2012. From the ongoing analysis of over 150 games involving (expert and non-expert, males and females, adults and child) judges, machines and hidden humans (foils for the machines), we present six particular conversations that took place between human judges and a hidden entity that (...) produced unexpected results. From this sample we focus on features of Turing’s machine intelligence test that the mathematician/code breaker did not consider in his examination for machine thinking: the subjective nature of attributing intelligence to another mind. (shrink)

Since Turing proposed the first test of intelligence, several modifications have been proposed with the aim of making Turing’s proposal more realistic and applicable in the search for artificial intelligence. In the modern context, it turns out that some of these definitions of intelligence and the corresponding tests merely measure computational power. Furthermore, in the framework of the original Turing test, for a system to prove itself to be intelligent, a certain amount of deceit is implicitly (...) required which can have serious security implications for future human societies. In this article, we propose a unified framework for developing intelligence tests which takes care of important ethical and practical issues. Our proposed framework has several important consequences. Firstly, it results in the suggestion that it is not possible to construct a single, context independent, intelligence test. Secondly, any measure of intelligence must have access to the process by which a problem is solved by the system under consideration and not merely the final solution. Finally, it requires an intelligent agent to be evolutionary in nature with the flexibility to explore new algorithms on its own. (shrink)

The Turing Test is one of the most disputed topics in artificial intelligence, philosophy of mind, and cognitive science. This paper is a review of the past 50 years of the Turing Test. Philosophical debates, practical developments and repercussions in related disciplines are all covered. We discuss Turing's ideas in detail and present the important comments that have been made on them. Within this context, behaviorism, consciousness, the 'other minds' problem, and similar topics in philosophy of (...) mind are discussed. We also cover the sociological and psychological aspects of the Turing Test. Finally, we look at the current situation and analyze programs that have been developed with the aim of passing the Turing Test. We conclude that the Turing Test has been, and will continue to be, an influential and controversial topic. -/- (This paper was reprinted in: The Turing Test: The Elusive Standard of Artificial Intelligence, J. H. Moor, ed., Kluwer Academic, pp. 23-78, 2003.). (shrink)

In 1950, Alan Turing proposed his eponymous test based on indistinguishability of verbal behavior as a replacement for the question "Can machines think?" Since then, two mutually contradictory but well-founded attitudes towards the Turing Test have arisen in the philosophical literature. On the one hand is the attitude that has become philosophical conventional wisdom, viz., that the Turing Test is hopelessly flawed as a sufficient condition for intelligence, while on the other hand is the overwhelming sense that (...) were a machine to pass a real live full-fledged Turing Test, it would be a sign of nothing but our orneriness to deny it the attribution of intelligence. The arguments against the sufficiency of the Turing Test for determining intelligence rely on showing that some extra conditions are logically necessary for intelligence beyond the behavioral properties exhibited by an agent under a Turing Test. Therefore, it cannot follow logically from passing a Turing Test that the agent is intelligent. I argue that these extra conditions can be revealed by the Turing Test, so long as we allow a very slight weakening of the criterion from one of logical proof to one of statistical proof under weak realizability assumptions. The argument depends on the notion of interactive proof developed in theoretical computer science, along with some simple physical facts that constrain the information capacity of agents. Crucially, the weakening is so slight as to make no conceivable difference from a practical standpoint. Thus, the Gordian knot between the two opposing views of the sufficiency of the Turing Test can be cut. (shrink)

In 2014, widespread reports in the popular media that a chatbot named Eugene Goostman had passed the Turing test became further grist for those who argue that the diversionary tactics of chatbots like Goostman and others, such as those who participate in the Loebner competition, are enabled by the open-ended dialog of the Turing test. Some claim a new kind of test of machine intelligence is needed, and one community has advanced the Winograd schema competition to address this (...) gap. We argue to the contrary that implicit in the Turing test is the cooperative challenge of using language to build a practical working understanding, necessitating a human interrogator to monitor and direct the conversation. We give examples which show that, because ambiguity in language is ubiquitous, open-ended conversation is not a flaw but rather the core challenge of the Turing test. We outline a statistical notion of practical working understanding that permits a reasonable amount of ambiguity, but nevertheless requires that ambiguity be resolved sufficiently for the agents to make progress. (shrink)

This paper aims to show that dominant conceptions of intelligence used in artificial intelligence (AI) are biased by normative assumptions that originate from the Global North, making it questionable if AI can be uncritically applied elsewhere without risking serious harm to vulnerable people. After the introduction in Sect. 1 we shortly present the history of IQ testing in Sect. 2, focusing on its multiple discriminatory biases. To determine how these biases came into existence, we define intelligence ontologically and underline its (...) constructed and culturally variable character. Turning to AI, specifically the Turing Test (TT), in Sect. 3, we critically examine its underlying intelligence conceptions. The test has been of central influence in AI research and remains an important point of orientation. We argue that both the test itself and how it is used in practice risk promoting a limited conception of intelligence which solely originated in the Global North. Hence, this conception should be critically assessed in relation to the different global contexts in which AI technologies are and will be used. In Sect. 4, we highlight how unequal power relations in AI research are a real threat, rather than just philosophical sophistry while considering the history of IQ testing and the TT’s practical biases. In the last section, we examine the limits of our account and identify fields for further investigation. Tracing colonial continuities in AI intelligence research, this paper points to a more diverse and historically aware approach to the design, development, and use of AI. (shrink)

This book provides a novel framework for understanding and revising labor markets and education policies in an era of machine learning. It posits that while learning and knowing both require thinking, learning is fundamentally different than knowing because it results in cognitive processes that change over time. Learning, in contrast to knowing, requires time and agency. Therefore, “learning algorithms”—that enable machines to modify their actions based on real-world experiences—are a fundamentally new form of artificial intelligence that have potential to be (...) even more disruptive to labor markets than prior introductions of digital technology. To explore the difference between knowing and learning, Turing’s “Imitation Game,”—that he proposed as a test for machine thinking—is expanded to include time dependence. The arguments presented in the book introduce three novel concepts: (1) Comparative learning advantage: This is a concept analogous to comparative labor advantage but arises from the disparate times required to learn new knowledge bases/skillsets. It is argued that in the future, comparative learning advantages between humans and machines will determine their division of labor. (2) Two dimensions of job performance—expertise and interpersonal: Job tasks can be sorted into two broad categories. Tasks that require expertise have stable endpoints, which makes these tasks inherently repetitive and subject to automation. Tasks that are interpersonal are highly context-dependent and lack stable endpoints, which makes these tasks inherently non-routine. Humans compared to machines have a comparative learning advantage along the interpersonal dimension, which is increasing in value economically. (3) The Learning Game is a time-dependent version of Turing’s “Imitation Game.” It is more than a thought experiment. The “Learning Game” provides a mathematical framework with quantitative criteria for training and assessing comparative learning advantages. The book is highly interdisciplinary—presenting philosophical arguments in economics, artificial intelligence, and education. It also provides data, mathematical analysis, and testable criteria that researchers in these fields will find of practical use. The book calls for a rethinking of how labor markets operate and how the education system should prepare students for future jobs. It concludes with a list of counterintuitive recommendations for future education and labor policies that all stakeholders—employers, employees, educators, students, and political leaders—should heed. (shrink)

In this paper, we look at the possibility of a machine having a sense of humour. In particular, we focus on actual machine utterances in Turing test discourses. In doing so, we do not consider the Turing test in depth and what this might mean for humanity, rather we merely look at cases in conversations when the output from a machine can be considered to be humorous. We link such outpourings with Turing’s “arguments from various disabilities” used (...) against the concept of a machine being able to think, taken from his seminal work of 1950. Finally we consider the role that humour might play in adding to the deception, integral to the Turing test, that a machine in practice appears to be a human. (shrink)

On a literal reading of `Computing Machinery and Intelligence'', Alan Turing presented not one, but two, practical tests to replace the question `Can machines think?'' He presented them as equivalent. I show here that the first test described in that much-discussed paper is in fact not equivalent to the second one, which has since become known as `the Turing Test''. The two tests can yield different results; it is the first, neglected test that provides the (...) more appropriate indication of intelligence. This is because the features of intelligence upon which it relies are resourcefulness and a critical attitude to one''s habitual responses; thus the test''s applicablity is not restricted to any particular species, nor does it presume any particular capacities. This is more appropriate because the question under consideration is what would count as machine intelligence. The first test realizes a possibility that philosophers have overlooked: a test that uses a human''s linguistic performance in setting an empirical test of intelligence, but does not make behavioral similarity to that performance the criterion of intelligence. Consequently, the first test is immune to many of the philosophical criticisms on the basis of which the (so-called) `Turing Test'' has been dismissed. (shrink)

This article offers comprehensive criticism of the Turing test and develops quality criteria for new artificial general intelligence (AGI) assessment tests. It is shown that the prerequisites A. Turing drew upon when reducing personality and human consciousness to “suitable branches of thought” re-flected the engineering level of his time. In fact, the Turing “imitation game” employed only symbolic communication and ignored the physical world. This paper suggests that by restricting thinking ability to symbolic systems alone (...) class='Hi'>Turing unknowingly constructed “the wall” that excludes any possi-bility of transition from a complex observable phenomenon to an abstract image or concept. It is, therefore, sensible to factor in new requirements for AI (artificial intelligence) maturity assessment when approaching the Tu-ring test. Such AI must support all forms of communication with a human being, and it should be able to comprehend abstract images and specify con-cepts as well as participate in social practices. (shrink)

What happens to philosophy when philosophical activities migrate to the AI lab? My article explores the philosophical work that has gone into the machine simulations of language and understanding after Alan Turing. The early experiments by AI practitioners such as Karen Spärck Jones, Richard Richens, Yorick Wilks, and others at the Cambridge Language Research Unit (CLRU) led to the creation of the machine interlingua, semantic networks, and other technological innovations central to the development of AI in the 1950s–1970s. I (...) attempt to show how, in the midst of their computational work, the CLRU pioneers engaged with Ludwig Wittgenstein, Ferdinand de Saussure, Rudolf Carnap, and other philosophers and developed startling new ways of formulating fundamental questions about language and human understanding. More significantly, their philosophical activities on the machine present an inclusive and culturally diverse picture of the world that profoundly negates the ethnocentric metaphysics of human-machine conundrums that John Searle and his critics represent in the Chinese Room debate. The familiar legacy of that debate has long distorted the narrative of AI origins through its simultaneous reiteration and repudiation of the Turing test. My study seeks to clarify those origins, but my primary goal is to demonstrate what it is like to practice philosophy on the machine and how the critique of metaphysics is made possible in the AI lab. (shrink)

This paper provides a survey of philosophical discussion of the "the Turing Test". In particular, it provides a very careful and thorough discussion of the famous 1950 paper that was published in Mind.

The Turing test has been studied and run as a controlled experiment and found to be underspecified and poorly designed. On the other hand, it has been defended and still attracts interest as a test for true artificial intelligence (AI). Scientists and philosophers regret the test’s current status, acknowledging that the situation is at odds with the intellectual standards of Turing’s works. This article refers to this as the Turing Test Dilemma, following the observation that the test (...) has been under discussion for over seventy years and still is widely seen as either too bad or too good to be a valuable experiment for AI. An argument that solves the dilemma is presented, which relies on reconstructing the Turing test as a thought experiment in the modern scientific tradition. It is argued that Turing’s exposition of the imitation game satisfies Mach’s characterization of the basic method of thought experiments and that Turing’s uses of his test satisfy Popper’s conception of the critical and heuristic uses of thought experiments and Kuhn’s association of thought experiments to conceptual change. It is emphasized how Turing methodically varied the imitation game design to address specific challenges posed to him by other thinkers and how his test illustrates a property of the phenomenon of intelligence and suggests a hypothesis on machine learning. This reconstruction of the Turing test provides a rapprochement to the conflicting views on its value in the literature. (shrink)

Turing''s test has been much misunderstood. Recently unpublished material by Turing casts fresh light on his thinking and dispels a number of philosophical myths concerning the Turing test. Properly understood, the Turing test withstands objections that are popularly believed to be fatal.

The Turing Test, originally proposed as a simple operational definition of intelligence, has now been with us for exactly half a century. It is safe to say that no other single article in computer science, and few other articles in science in general, have generated so much discussion. The present article chronicles the comments and controversy surrounding Turing's classic article from its publication to the present. The changing perception of the Turing Test over the last fifty years (...) has paralleled the changing attitudes in the scientific community towards artificial intelligence: from the unbridled optimism of 1960's to the current realization of the immense difficulties that still lie ahead. I conclude with the prediction that the Turing Test will remain important, not only as a landmark in the history of the development of intelligent machines, but also with real relevance to future generations of people living in a world in which the cognitive capacities of machines will be vastly greater than they are now. (shrink)

Stuart M. Shieber’s name is well known to computational linguists for his research and to computer scientists more generally for his debate on the Loebner Turing Test competition, which appeared a decade earlier in Communications of the ACM. 1 With this collection, I expect it to become equally well known to philosophers.

I have discussed the frame problem and the Turing test at length, but I have not attempted to spell out what I think the implications of the frame problem ...

The Turing Test is claimed by many to be a way to test for the presence, in computers, of such ``deep'' phenomena as thought and consciousness. Unfortunately, attempts to build computational systems able to pass TT have devolved into shallow symbol manipulation designed to, by hook or by crook, trick. The human creators of such systems know all too well that they have merely tried to fool those people who interact with their systems into believing that these systems really (...) have minds. And the problem is fundamental: the structure of the TT is such as to cultivate tricksters. A better test is one that insists on a certain restrictive epistemic relation between an artificial agent A, its output o, and the human architect H of A – a relation which, roughly speaking, obtains when H cannot account for how A produced o. We call this test the ``Lovelace Test'' in honor of Lady Lovelace, who believed that only when computers originate things should they be believed to have minds. (shrink)

I advocate a theory of syntactic semantics as a way of understanding how computers can think (and how the Chinese-Room-Argument objection to the Turing Test can be overcome): (1) Semantics, considered as the study of relations between symbols and meanings, can be turned into syntax – a study of relations among symbols (including meanings) – and hence syntax (i.e., symbol manipulation) can suffice for the semantical enterprise (contra Searle). (2) Semantics, considered as the process of understanding one domain (by (...) modeling it) in terms of another, can be viewed recursively: The base case of semantic understanding –understanding a domain in terms of itself – is syntactic understanding. (3) An internal (or narrow ), first-person point of view makes an external (or wide ), third-person point of view otiose for purposes of understanding cognition. (shrink)

In this paper we discuss the views on the Turing test of four influential thinkers who belong to the tradition of analytic philosophy: Ludwig Wittgenstein, Noam Chomsky, Hilary Putnam and John Searle. Based on various beliefs about philosophical and/or linguistic matters, they arrive at different assessments of both the significance and suitability of the imitation game for the development of cognitive science and AI models. Nevertheless, they share a rejection of the idea that one can treat Turing test (...) as a test for “machine thinking.” This seems to stem from a concern for the proper use of language —one that is a fundamental methodological commitment of analytic philosophy. (shrink)

In the classical Turing test, participants are challenged to tell whether they are interacting with another human being or with a machine. The way the interaction takes place is not direct, but a distant conversation through computer screen messages. Basic forms of interaction are face-to-face and embodied, context-dependent and based on the detection of reciprocal sensorimotor contingencies. Our idea is that interaction detection requires the integration of proprioceptive and interoceptive patterns with sensorimotor patterns, within quite short time lapses, so (...) that they appear as mutually contingent, as reciprocal. In other words, the experience of interaction takes place when sensorimotor patterns are contingent upon one’s own movements, and vice versa. I react to your movement, you react to mine. When I notice both components, I come to experience an interaction. Therefore, we designed a “minimal” Turing test to investigate how much information is required to detect these reciprocal sensorimotor contingencies. Using a new version of the perceptual crossing paradigm, we tested whether participants resorted to interaction detection to tell apart human from machine agents in repeated encounters with these agents. In two studies, we presented participants with movements of a human agent, either online or offline, and movements of a computerized oscillatory agent in three different blocks. In each block, either auditory or audiovisual feedback was provided along each trial. Analysis of participants’ explicit responses and of the implicit information subsumed in the dynamics of their series will reveal evidence that participants use the reciprocal sensorimotor contingencies within short time windows. For a machine to pass this minimal Turing test, it should be able to generate this sort of reciprocal contingencies. (shrink)

The Turing Test is routinely understood as a behaviourist test for machine intelligence. Diane Proudfoot has argued for an alternative interpretation. According to Proudfoot, Turing’s claim that intelligence is what he calls ‘an emotional concept’ indicates that he conceived of intelligence in response-dependence terms. As she puts it: ‘Turing’s criterion for “thinking” is…: x is intelligent if in the actual world, in an unrestricted computer-imitates-human game, x appears intelligent to an average interrogator’. The role of the famous (...) test is thus to provide the conditions in which to examine the average interrogator’s responses. I shall argue that Proudfoot’s analysis falls short. The philosophical literature contains two main models of response-dependence, what I shall call the transparency model and the reference-fixing model. Proudfoot resists the thought that Turing might have endorsed one of these models to the exclusion of the other. But the details of her own analysis indicate that she is, in fact, committed to the claim that Turing’s account of intelligence is grounded in a transparency model, rather than a reference-fixing one. By contrast, I shall argue that while Turing did indeed conceive of intelligence in response-dependence terms, his account is grounded in a reference-fixing model, rather than a transparency one. This is fortunate, because, as an account of intelligence, the transparency model is arguably problematic in a way that the reference-fixing model isn’t. (shrink)

The paper examines the nature of the behavioral evidence underlying attributions of intelligence in the case of human beings, and how this might be extended to other kinds of cognitive system, in the spirit of the original Turing Test. I consider Harnad's Total Turing Test, which involves successful performance of both linguistic and robotic behavior, and which is often thought to incorporate the very same range of empirical data that is available in the human case. However, I argue (...) that the TTT is still too weak, because it only tests the capabilities of particular tokens within a preexisting context of intelligent behavior. What is needed is a test of the cognitive type, as manifested through a number of exemplary tokens, in order to confirm that the cognitive type is able to produce the context of intelligent behavior presupposed by tests such as the TT and TTT. (shrink)

This paper discusses some difficulties in understanding the Turing test. It emphasizes the importance of distinguishing between conceptual and empirical perspectives and highlights the former as introducing more serious problems for the TT. Some objections against the Turingian framework stemming from the later Wittgenstein’s philosophy are exposed. The following serious problems are examined: 1) It considers a unique and exclusive criterion for thinking which amounts to their identification; 2) it misidentifies the relationship of speaking to thinking as that of (...) a criterion; 3) it neglects the “natural” course of the development in semantics. However, these considerations suggest only that it is problematic to label a successful chatbot as a “thinking entity” without further qualifications, but not necessarily and once and for all incorrect. Philosophy has only little to say about the technical possibility of creating such an effective program. (shrink)

Alan Turing, einer der Gründerväter der modernen Informatik, diskutierte in seinem 1950 veröffentlichten Artikel »Computing Machinery and Intelligence« die Frage, ob Maschinen denken können. Dies wirft jedoch die schwierige Frage auf, was Denken denn sei. Um diese Frage zu umgehen, schlägt Turing vor, stattdessen eine Frage zu stellen, die sich durch bloße Beobachtung klären lässt, nämlich ob eine Maschine ein bestimmtes Spiel erfolgreich spielen könne.

This paper argues against the moral Turing test as a framework for evaluating the moral performance of autonomous systems. Though the term has been carefully introduced, considered, and cautioned about in previous discussions :251–261, 2000; Allen and Wallach 2009), it has lingered on as a touchstone for developing computational approaches to moral reasoning :98–109, 2015). While these efforts have not led to the detailed development of an MTT, they nonetheless retain the idea to discuss what kinds of action and (...) reasoning should be demanded of autonomous systems. We explore the flawed basis of an MTT in imitation, even one based on scenarios of morally accountable actions. MTT-based evaluations are vulnerable to deception, inadequate reasoning, and inferior moral performance vis a vis a system’s capabilities. We propose verification—which demands the design of transparent, accountable processes of reasoning that reliably prefigure the performance of autonomous systems—serves as a superior framework for both designer and system alike. As autonomous social robots in particular take on an increasing range of critical roles within society, we conclude that verification offers an essential, albeit challenging, moral measure of their design and performance. (shrink)

The main factor of intelligence is defined as the ability tocomprehend, formalising this ability with the help of new constructsbased on descriptional complexity. The result is a comprehension test,or C- test, which is exclusively defined in computational terms. Due toits absolute and non-anthropomorphic character, it is equally applicableto both humans and non-humans. Moreover, it correlates with classicalpsychometric tests, thus establishing the first firm connection betweeninformation theoretical notions and traditional IQ tests. The TuringTest is compared with the C- test (...) and the combination of the two isquestioned. In consequence, the idea of using the Turing Test as apractical test of intelligence should be surpassed, and substituted bycomputational and factorial tests of different cognitive abilities, amuch more useful approach for artificial intelligence progress and formany other intriguing questions that present themselves beyond theTuring Test. (shrink)

The Turing Test is best regarded as a model to test for intelligence, where an entity’s intelligence is inferred from its ability to be attributed with ‘human-likeness’ during a text-based conversation. The problem with this model, however, is that it does not care if or how well an entity produces a meaningful conversation, as long as its interactions are humanlike enough. As a consequence, the TT attracts projects that concentrate on how best to fool the judges. In light of (...) this, I propose a new version of the TT: the Questioning Turing Test. Here, the entity has to produce an enquiry rather than a conversation; and it is parametrised along two further dimensions in addition to ‘human-likeness’: ‘correctness’, evaluating if the entity accomplishes the enquiry; and ‘strategicness’, evaluating how well the entity accomplishes the enquiry, in terms of the number of questions asked. (shrink)

Brain damage can cause massive changes in consciousness levels. From a clinical and ethical point of view it is desirable to assess the level of residual consciousness in unresponsive patients. However, no direct measure of consciousness exists, so we run into the philosophical problem of other minds. Neurologists often make implicit use of a Turing test-like procedure in an attempt to gain access to damaged minds, by monitoring and interpreting neurobehavioral responses. New brain imaging techniques are now being developed (...) that permit communication with unresponsive patients, using their brain signals as carriers of messages relating to their mental states. (shrink)

This dissertation examines aspects of the interplay between computing and scientific practice. The appropriate foundational framework for such an endeavour is rather real computability than the classical computability theory. This is so because physical sciences, engineering, and applied mathematics mostly employ functions defined in continuous domains. But, contrary to the case of computation over natural numbers, there is no universally accepted framework for real computation; rather, there are two incompatible approaches --computable analysis and BSS model--, both claiming to formalise algorithmic (...) computation and to offer foundations for scientific computing. The dissertation consists of three parts. In the first part, we examine what notion of 'algorithmic computation' underlies each approach and how it is respectively formalised. It is argued that the very existence of the two rival frameworks indicates that 'algorithm' is not one unique concept in mathematics, but it is used in more than one way. We test this hypothesis for consistency with mathematical practice as well as with key foundational works that aim to define the term. As a result, new connections between certain subfields of mathematics and computer science are drawn, and a distinction between 'algorithms' and 'effective procedures' is proposed. In the second part, we focus on the second goal of the two rival approaches to real computation; namely, to provide foundations for scientific computing. We examine both frameworks in detail, what idealisations they employ, and how they relate to floating-point arithmetic systems used in real computers. We explore limitations and advantages of both frameworks, and answer questions about which one is preferable for computational modelling and which one for addressing general computability issues. In the third part, analog computing and its relation to analogue (physical) modelling in science are investigated. Based on some paradigmatic cases of the former, a certain view about the nature of computation is defended, and the indispensable role of representation in it is emphasized and accounted for. We also propose a novel account of the distinction between analog and digital computation and, based on it, we compare analog computational modelling to physical modelling. It is concluded that the two practices, despite their apparent similarities, are orthogonal. (shrink)