Skip to main content

Advertisement

SpringerLink
Log in

Hidden Interlocutor Misidentification in Practical Turing Tests

  • Published:
Minds and Machines Aims and scope Submit manuscript

Abstract

Based on insufficient evidence, and inadequate research, Floridi and his students report inaccuracies and draw false conclusions in their Minds and Machines evaluation, which this paper aims to clarify. Acting as invited judges, Floridi et al. participated in nine, of the ninety-six, Turing tests staged in the finals of the 18th Loebner Prize for Artificial Intelligence in October 2008. From the transcripts it appears that they used power over solidarity as an interrogation technique. As a result, they were fooled on several occasions into believing that a machine was a human and that a human was a machine. Worse still, they did not realise their mistake. This resulted in a combined correct identification rate of less than 56%. In their paper they assumed that they had made correct identifications when they in fact had been incorrect.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Loebner Prize homepage: http://www.loebner.net/Prizef/loebner-prize.html retrieved Monday October 13, 2009: 12.12.

  2. Philosophy of Information Blog—“The Loebner Prize from a judge’s perspective” (Monday October 13, 2008) retrieved Monday 19 October 2009: 12.13.

  3. Marc Allen Turing Test Protocol: http://www.1bdi.co.uk/matt/ retrieved Sunday 25 October 2009: 14.49.

  4. Alicebot: http://alicebot.blogspot.com/ retrieved Monday 19 october 2009: 15.42.

  5. Elbot: http://www.elbot.com/ retrieved Wednesday 28 October 2009: 11.26 am.

  6. .. average interrogator would not have more than 70% chance of making the right identification after 5 min of questioning (Turing 1950, p. 442).

  7. http://www.artificial-solutions.com/ accessed Monday 19 October 2009: 17.18.

References

  • Carpenter, R. (2009). Jabberwacky: Communication, companionship intelligence. http://www.jabberwacky.com/. Accessed October 25, 2009.

  • Floridi, L., Taddeo, M., & Turilli, M. (2009). Turing’s imitation game—Still an impossible challenge for all machines and some judges. An Evaluation of the 2008 Loebner Contest. Minds and Machines, 19(1), 145–150.

    Article  Google Scholar 

  • Loebner Prize (2008). Loebner prize for artificial intelligence—Home of the first Turing test. http://www.loebner.net/Prizef/. Accessed October 29, 2009.

  • Roberts, F. & Gulsdorff, B. (2007). IVA2007—LNAI 4722, pp. 420–421.

  • Roberts, F. (2005). The AI of elbot. Unpublished.

  • Roberts, F. (2008). A social psychological approach to dialogue simulation. Unpublished.

  • Shah, H., & Henry, O. (2005). The confederate effect in human–machine textual interaction. In A. Zemliak & N. Mastorakis (Eds.), Proceedings of the 5th WSEAS international conference on information science, communications and applications (ISCA 2005), Cancun, Mexico, May 11–14, ISBN: 960-8457-22-X, pp. 109–114.

  • Shah, H., & Pavlika, V. (2005). Text-based dialogical e-query systems: Gimmick or convenience? Proceedings of the 10th international conference on speech and computers (SPECOM), Patras, Greece, October 17–19, ISBN: 5-7452-0110-X, Vol. II, pp. 425–428.

  • Shah, H., & Warwick, K. (2009). Emotion in the Turing test: A downward trend for machines in recent Loebner prizes. Chapter XVII (Section V). In J. Vallverdú & D. Casacuberta (Eds.), Handbook of research on synthetic emotions and sociable robotics: New applications in affective computing and artificial intelligence. USA: Information Science Reference, ISBN: 978-1-60566-354-8.

  • Shah, H., & Warwick, K. (2010). Testing Turing’s five minutes parallel-paired imitation game. Kybernetes, 39(3), 449–465.

    Google Scholar 

  • Turing, A. M., Braithwaite, R., Jefferson, G., & Newman, M. (1952). Can automatic calculating machines be said to think? In J. Copeland (Eds.), The essential Turing—The ideas that gave birth to the computer age (pp. 487–506). Oxford: Clarendon Press.

    Google Scholar 

  • Turing, A. M. (1950). Computing, machinery and intelligence. Mind, LIX(236), 433–460.

  • Turing, A. M. (1948). Intelligent machinery. In B. J. Copeland (Ed.), The essential Turing—The ideas that gave birth to the computer age. Oxford: Clarendon Press, 2004.

  • Turkel, S. (1997). Life on the screen-identity in the age of the internet. London: Pheonix Paperback.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Systems Engineering, The University of Reading, Whiteknights, Reading, Berkshire, RG6 6AY, UK

    Huma Shah & Kevin Warwick

Authors
  1. Huma Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin Warwick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huma Shah.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shah, H., Warwick, K. Hidden Interlocutor Misidentification in Practical Turing Tests. Minds & Machines 20, 441–454 (2010). https://doi.org/10.1007/s11023-010-9219-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11023-010-9219-6

Keywords

Search

Navigation