Subject: Re: [extropy-chat] Engineered Religion

From: justin corwin <[email-masked]>
Date: Mon, 21 Mar 2005 13:02:36 -0800

>"No child of mine will ever cower before an imaginary God. It is
beneath the dignity of >human beings and it is beneath the dignity of
our descendants. If the lightning is beautiful, >then let us see the
beauty in electricity without need for thunder deities; for if we
cannot >learn to take joy in the merely real, our lives will be empty
indeed."

~Eliezer Yudkowsky

One great fear of mine is the tyrrany of sounding good. Eliezer is a
prime example of a person who has tuned and tuned in the search for
rationality. Unfortunately, I suspect, there is a class of
'improvements' one can make which correspond more to obscuring flaws
than making true statements. It is a problem I have noted myself on
more than one occasion. Here Eliezer is responding to a theist who
complained that we need to instill our mind children with some kind of
religion that will force them to respect their elders, lest we be
obsoleted. He continues:

>"But I'm not going to try to hardcode that, not in a child nor in an
AI. As an atheist, I have >a simple, matter-of-fact confidence that
religionists once had and relinquished long ago. I >don't think I need
to load the dice for my answer to win. All I need is to set in motion
the >dynamics that seek truth, i.e., some computable approximation of
Solomonoff induction. If >there were the tiniest shred of truth to
religion, that would be enough to uncover it. If you >have even a
droplet of honest belief left, not just empty excuses for a faith you
lost long >ago, you will not ask me to load an AI's dice in favor of
your pet theory. Let the truth win >out."

All true statements. All very appealing (at least to this rationalist
and this truthseeker). But, the subtle shift in conversation here is
quite nearly unnoticed. We've transitioned to instilling beliefs in a
mind, to better them and ourselves, to talking about the structure of
the mind, to fixing it so there is only one answer. Perhaps because
the theist is muddled in his thinking this blanket approach is valid.
It's true that Eliezer's objections do entirely refute John C Wright's
theistic aspirations. But his argument does not directly address his
points.

A general question: What is intellectual honesty? Eliezer has a real
commitment to truth. However, and I fear this is a general point,
being committed to truth is not sufficient. Eliezer in this example,
and others in many examples (I choose Eliezer because I believe he's
not making any other mistakes here) has changed the context, the
discussion has been shifted to allow him his total commitment to
certainty. By changing the context slightly he's found a place where
he can shoot down this theistic argument with perfect aplomb and sound
like a hero. But is he? He's making arguments that are true, and
insightful(even poetic, perhaps) but they aren't in the original exact
vein of discussion. Isn't that somewhat misleading? Or am I making
something of a molehill? Perhaps Eliezer has simply reframed the
question in general terms, much as I'm generalizing his statements for
logical effect.

Let's continue in that vein, and move reducto ad absurdum. Suppose a
fully rational, truthful being, that only chooses to engage in
discussion when certain, and always seeks to twist contexts to those
he's more comfortable in, to the limits of his self respect and
intellectual honesty. Luckily, mythology is replete with examples of
this type. The Zen Master, the Oracle, Yoda, all inscrutable
characters who are right, and insightful, and powerful creatures, but
maddening, because they only rouse themselves to croak factually
accurate and unassailable arguments, and refuse to engage in fringe
discussion.

There are two factors here. One is the very real problem of authority
acceptance. Many self-aware Masters rage at their disciples on both
sides, chiding them for accepting the Master's word without question,
and also being annoyed when they don't recognize and internalize the
truth the Master offers them. So the good master retreats into
relative silence to avoid corrupting and doing a disservice to all
those who listen to him. Speaking when certain, and able to tell how
his words will affect. This admirable strategy is always blended with
persona maintenence, a despicable practice of hiding, changing, and
sculpting information to maintain certain relationships and
reputations. Shame on the Master who can't bear to have students see
him wrong.

The second factor is subtler, and the one I have been trying to
explore above. The Master categorizes within his subject. He divides
the realm of his understanding by function or taxonomy, he asks
questions and answers with statements which exist along those lines.
The Fool asks sweeping, conjoined questions, stabs at understanding
that smears across the subject. The Master, presented with these
questions, maps the question to his understanding, finding pieces of
it within some division of his knowledge, addresses this part (perhaps
rightly) believing himself to have dealt with the entirety. After all,
a single contradiction is all you need to invalidate an entire
argument.

I don't know whether this represents an important distinction when
you're just learning to be a rationalist. I haven't yet reached the
point where it even constitutes a significant portion of my mistakes.
But it is A mistake.

I know there is a rank above that of Inscrutable Master. I don't have
all the details yet, but he's humble and detailed and truthful. She
answers questions in the spirit they are asked, but as correctly as
she knows how. He presents his uncertainty, his incomplete scraps of
knowledge, and his current thinking, because it too, is information.
And she never takes an easy win, when there are more interesting and
informative portions of an argument. Noisy Errors are to be preferred.

-- 
Justin Corwin
[email-masked]
http://outlawpoet.blogspot.com
http://www.adaptiveai.com
_______________________________________________
extropy-chat mailing list
[email-masked]
http://lists.extropy.org/mailman/listinfo/extropy-chat

---

From: Jef Allbright <[email-masked]>
Date: Mon, 21 Mar 2005 16:10:31 -0800

Justin -

You make a very important point, however one that is all too easily 
filtered out by the very mechanism that it seeks to illuminate.

Each of us is limited by the context within which we perceive and 
process our individual approximation of the "truth".  The laser focus 
with which many of us are most comfortable tends to ignore the larger 
context that encompasses both viewpoints, and how they both fit, 
perfectly, within the greatest context that we think of as reality.  
Paradox is always a case of insufficient context.

    * Is is relatively easy to find weaknesses in another's structure of
      thought and with laser accuracy, illuminate the errors and prove
      them wrong.

    * It is much more challenging, and more rewarding, to cooperate in
      discovering the context in which another's belief is held, and
      then help build a larger context in which growth is facilitated.

    * More challenging yet, is to appreciate the limits of one's own
      context of thought, and be able to assign pragmatic probabilities
      as to its permanence. Interestingly, while it is impossible to see
      one's own context directly, as from outside oneself, it is
      possible in a second order way based on accumulating experience of
      self over time, to tune this probability in the proper direction.

Rationality, in all its aspects, is a powerful capability, but it's 
worth keeping in mind that it always operates within the bounds of 
incomplete knowledge and computational resources, and remains embedded 
within the larger, older, and successful structure of the environment 
within which we evolved.

- Jef
http://www.jefallbright.net

---

From: "Eliezer S. Yudkowsky" <[email-masked]>
Date: Mon, 21 Mar 2005 17:14:52 -0800

> theistic aspirations. But his argument does not directly address his

Huh?  I addressed the two main points Wright had, as I saw them:

1)  Wright wants to program in religion as fixed.  I regard religion as 
possessing and relying on factual components which would be invalidated 
by a simple truthseeking dynamic.  Programming in fixed beliefs creates 
a conflict of interest over whose pet belief gets programmed; 
programming in a truthseeking dynamic without loaded dice seems to me a 
fair resolution.  C.f. http://sl4.org/wiki/CollectiveVolition, 
_Motivations_, "5. Avoid creating a motive for modern-day humans to 
fight over the initial dynamic."

2)  Wright has warm and fuzzy feelings about daddygods whose imaginary 
threat of eternal hell keeps people in line.  I reply with my warm and 
fuzzy feelings about free and independent humans, living without fear, 
moral because that is who they choose to be.

If you feel I failed to address a point, why do you not state clearly 
what it is?  Sheeze, and they call me Yoda.  I never signed up to be 
your Zen Master.  I just ordinarily talk like that, y'know, it's the way 
I express my philosophy of being human.

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
_______________________________________________
extropy-chat mailing list
[email-masked]
http://lists.extropy.org/mailman/listinfo/extropy-chat

---

From: "Eliezer S. Yudkowsky" <[email-masked]>
Date: Mon, 21 Mar 2005 17:21:56 -0800

> theistic aspirations. But his argument does not directly address his

So is the point I'm not addressing the idea that we must program AIs 
with a modern-day human religion that advocates compassion because 
otherwise they'll wipe us out?  I guess I didn't address this point 
because it doesn't interest me.  The answer is "That's not how 
Friendliness engineering works!", and you, Corwin, can probably 
explicate why, without me going into a lecture on things that I've said 
before only around ten million times.