Subject: Re: [ExI] Unfrendly AI is a mistaken idea.

From: Eugen Leitl <[email-masked]>
Date: Thu, 24 May 2007 12:19:48 +0200

On Thu, May 24, 2007 at 08:12:19PM +1000, Stathis Papaioannou wrote:

>    Emotion is linked to motivation, not intelligence per se. Intelligence
>    is an ability, like being able to lift heavy things. The ability to
>    lift heavy things would never have evolved naturally without an
>    associated motivation to do so, but we build powerful lifting machines
>    that would sit there rusting if we didn't provide motivation for them
>    to do their thing. Similarly, there is nothing contradictory in a
>    machine capable of fantastically complex cognitive feats that would
>    just sit there inertly unless specifically offered a problem, and then
>    solve the problem as an intellectual exercise, completely
>    disinterested in any practical applications. There is even a model for
>    this in human mental illness: patients with so-called negative
>    symptoms of schizophrenia can be cognitively and physically intact,
>    but lack motivation and the ability to experience emotion.

The critical points here are:

1) can we construct such mentally defect artifical agents, before
   we can build the other kind?

2) is a population of such a stable system?

3) is it a good idea?

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
_______________________________________________
extropy-chat mailing list
[email-masked]
http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat

---

From: Eugen Leitl <[email-masked]>
Date: Thu, 24 May 2007 22:11:49 +0200

On Thu, May 24, 2007 at 12:43:49PM -0700, Samantha Atkins wrote:

> I believe I see a basic assumption that humans are and always will be 
> more trustworthy (moral?)  than non-biological intelligences.  

If the easiest way to build one is to use a human primate for
a blueprint, both need not to be that different.

I also see a problem with building artificial agents with no
designated empathy with bipedal primates built-in. It looks
like begging for disaster.