Turing tests

[pmb]

When people aren't trying to screen out people vs. non-people, then we have many many instances of computers successfully passing themselves off as people and having long conversations with unwitting strangers who never caught on. If I told you that a particular AIM account was actually a perl script designed to pass the Turing test, and, when you initiated a chat with that account it said "No, man. That's just one of my friends playing a trick on me - I'm totally real, and that's totally a hoax", how could it convince you of its humanity without resorting to out-of-band methods "call me on the phone" or "check out my webpage"?

If you can't think of a method, then I submit that computers have already passed the Turing test.

Comments

[amoken.livejournal.com]

I cannot think of a standardizable test, nor could I give you a time limit on how long a test could take. I think it's indeterminate, and a moving target anyway. But for starters, I'd go through the random sorts of stuff that computers tend to have difficulty with (common sense—when Alice goes to the store does her head go with her; amateur chatter in several domains; mindless chitchat; tell me about yourself, your parents, your home, your job, etc; and so on), ask more in-depth questions in response, and possibly go through some of the things computers tend to excel at.

[pmb.livejournal.com]

Right, but I contend that the person (who is irritated with the prank) would not give you the in depth answers you want - they've had enough deep probing from strangers. And I'm pretty sure I could write a program that answered most facile questions, and made pleasant chitchat and then quickly became pissed off.

Our biggest Turing Test successes have been in simulating conversations as people who have mental problems - paranoid schizophrenics turn out to be the easiest ones of all. I think irritation from a bunch of people asking if you are really human might lead to a simulatable frame of mind (getting pissed off when the answer isn't obvious for example).

[misuba.livejournal.com]

It's not so much that computers have passed the Turing test, as that we've failed it. Most of the time we don't communicate with each other in a way that really expresses our humanity.

[pmb.livejournal.com]

That's a neat perspective.

[amoken.livejournal.com]

Oh! If they claim to be intentionally uncooperative, then sure. But then we get to the point where I don't care, cuz it's not a very interesting game for me. :)

[amoken.livejournal.com]

I'd submit that unless you're fairly actively looking for a mate (there are probably a couple other situations), species tends to be irrelevant, and we generally try to avoid communicating irrelevant info. Maybe you are communicating with a machine (or a cat! whatever), but if it has the information you want, likely it doesn't matter. It's fabulous!

[pmb.livejournal.com]

Right! But the being-unable-to-tell phenomenon is new and, to my mind, unprecedented and cool.

[pmb.livejournal.com]

Put more succinctly, we've moved from "on the internet nobody knows if you are a dog" to "on the internet nobody CARES if you are a dog".

Whoah.

[amoken.livejournal.com]

Whoah.

Hey, now, I'm not a horse! I promise!

[mrsmalkav.livejournal.com]

at peoplelink, when i worked customer service, i'd get random "a/s/l? wanna cyber?" messages. one day, i decided to pipe the eliza bot that we had into the conversation.

the guy got more and more aggravated, but he never questioned her being a real person.

"do you like choclit?" We were discussing you, not me. What made you think of that?
"do you like it all over your body?" Oh yes... like it all over my body!

AWESOME

[pmb.livejournal.com]

THAT IS THE KIND OF RESEARCH THAT OUR DISCIPLINE NEEDS SOOOOOO BAD

[mycrust.livejournal.com]

I was going to say something similar, but you beat me to it.

[patrissimo.livejournal.com]

but then they haven't passed the turing test. They have rendered an unconclusive answer. Or rather, once computers are good enough to simulate a human who has gotten annoyed, we need to make sure that we don't judge "looks like an annoyed human" as passing the Turing Test, we judge it as an ambiguous answer.

I don't think that means that computers have passed the turing test. It just means that we were giving a weak test because computers were so bad.

Look, by a similar argument to yours, the null response could be considered "passing the turing test", because hell, it could just be a person who doesn't feel like talking. A reasonable definition for the turing test has to include "After talking co-operatively for as long as the interviewer desires..." Heck, it might, for all I know. Turing was a smart fella.

[ouro.livejournal.com]

I'm not so sure that the simulation of a deranged interlocutor can be considered a valid pass of the Turing Test. More interesting, then, would be able to simulate a number of moods and mood shifts in the conversation.

Passing a different kind of scrutiny than "prove it"

[inkandessence.livejournal.com]

In the original form of Turing's imitation game, the goal wasn't to come up with clever answers or attitude towards "Prove you are human! Now!" - it was to secretly pass as human while being questioned by an unwitting examiner about some other area of human life, e.g. the experience of being a woman.

That was why TheGuessingGame (http://www.theguessinggame.net/) made the news recently (http://en.wikinews.org/wiki/Simon's_Rock_College_tests_Alan_Turing_theories_with_'Imitation_Game'_experiment)

The confusion is caused by the Loebner Prize (http://www.loebner.net/Prizef/loebner-prize.html), which takes a more direct "prove it!" attitude. But I think the important thing to remember is what the Turing Test was supposed to measure - intelligence. Turing wasn't interested in whether a computer could convincingly blow you off - a tape recorder wired to a doorbell can convincingly blow you off. He was interested in whether a computer could simulate a short, free-ranging conversation in a way that created the impression of intelligence.

[olstad.livejournal.com]

I tend to use a lot of colloquialisms and punny conversation. I've never used AIM and I haven't used ytalk in a long time, but I think that's true in my written conversation as well... are AIs up to speed on such things these days?

[bettsbaby.livejournal.com]

Being unfamiliar with this...have computers already started co-opting the "you know when you" kind of language? (i.e. You know when you go to the supermarket and there's a guy in the 10 items or less checkout line with 20 items? Don't you hate that?) The language that a lot of people use to try to make a connection between themselves and the person they are speaking with, instead of just speaking from their own experience. (i.e. When I go to the supermarket I hate it when there's someone in the 10 items or less checkout line with 20 items)

[pmb.livejournal.com]

Computers aren't very funny, but they pick up colloquialisms well - after all, all they know is what you tell them, so tell them your colloquialisms once, and they will know them forever.

Their wordplay abilities tend to be directly proportional to the amount of data on which they were trained, and inversely proportional to the breadth of subject material they are trained on. Also, puns generally require some knowledge of pronunciation and the semantics of the area under discussion - both of which computers are simply abysmal at.

Also, they do pretty much suck at chatting, it's just that our requirements for chatting online are so low. Chat conersations are atrocious - you think you've seen bad grammar and spelling and annoying abbreviations in emails and livejournal? You ain't seen nothing compared to the grammarial wasteland that is IM. So perhaps it's like Misuba said above - it's not that they could pass it, it's that we would fail it.

Re: Passing a different kind of scrutiny than "prove it"

[pmb.livejournal.com]

Thank you for that clarification!

That almost makes me more convinced that computers could pass. I'll make sure to follow the results from that Simon's Rock test...

[pmb.livejournal.com]

Hrm. It depends on what you mean. Computers tend to talk like the people they are trained on, and rhetorical questions and other devices like that are seldom employed in online chat - they are probably too subtle.

[akjdg.livejournal.com]

I'm pretty clueless on the topic also, but I would initially think that focusing on arcane knowledge or logical leaps that are heavily dependent on specific knowledge would be effective.

But if the algorithm had the opportunity to crawl the web and learn 'everything', and meaningfully digest it, then that might not be a useful approach. The web is far more knowledgeable (and disknowledgeable) than I.

So. Beats me.

In other news, You'll be delighted to know that I'm in Iowa and mailed 26 cows today. My herd was up to nearly 200, but we hit Des Moines and Amy's quick eyes saw several fast food 'restaurants'.

Way too appropriate!

[dragonmudd.livejournal.com]

http://www.overheardinnewyork.com/archives/001936.html

Especially read the subject line.