Subject: Re: [extropy-chat] Engineered Religion-- Your Mom and the Machine

From: Robin Hanson <[email-masked]>
Date: Fri, 25 Mar 2005 14:10:01 -0500

At 12:45 PM 3/25/2005, [email-masked] wrote:
>Ben, whose sense of humor is no better and no worse than my own, poses a
>significant epistimological question. He is asking on what grounds the 
>faithful
>(as we call ourselves) or foam-at-the-mouth zealots in teapot-shaped hats (as
>others call us) take certain things on faith, or on authority, or as a 
>result of
>innate knowledge or revelation? How does one distinguish a valid authority 
>from
>an invalid, or tell a true intuition from a false?
>I cannot imagine that the majority of the posters here would be interested 
>in a
>discussion of this kind, which is a straight-up philosophical question, black
>without cream or sugar. I will be happy to write him privately with my answer,
>such as it is.

I'll bet there are in fact enough of us interested in that sort of 
conversation to
make it worth having here.  I know I am, at least if it is at a high level.

I don't see why your machine needs to have a "higher rule" than empiricism 
so that
it will not destroy the world.  I just has to not *want* to destroy the 
world.
It doesn't need to believe in morality, or consciousness, or a "Mother rule".
You seem to imagine that certain wants are natural and dangerous, and so 
creatures
must be built to follow Mother rules which overrule those natural wants or all
may be lost.  I can see a concern about the wants that evolutionary selection
would produce, but if we are just talking about creating a single creature 
I don't
see why it can't be just made not to want to destroy the world.

Robin Hanson  [email-masked]  http://hanson.gmu.edu
Assistant Professor of Economics, George Mason University
MSN 1D3, Carow Hall, Fairfax VA 22030-4444
703-993-2326  FAX: 703-993-2323 

_______________________________________________
extropy-chat mailing list
[email-masked]
http://lists.extropy.org/mailman/listinfo/extropy-chat

---

From: Eugen Leitl <[email-masked]>
Date: Fri, 25 Mar 2005 21:02:03 +0100

> see why it can't be just made not to want to destroy the world.

Can you write down all the ways of not destroying the world? Or, easier, its
negative?

All a priori, of course.

(People as sensors are no fair, since entangled in the measurement).

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a>
______________________________________________________________
ICBM: 48.07078, 11.61144            http://www.leitl.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
http://moleculardevices.org         http://nanomachines.net
_______________________________________________
extropy-chat mailing list
[email-masked]
http://lists.extropy.org/mailman/listinfo/extropy-chat

---

From: "Eliezer S. Yudkowsky" <[email-masked]>
Date: Fri, 25 Mar 2005 12:58:46 -0800

[email-masked] wrote:
> 
> To the other posters I will ask a question related to the one which started the
> thread:
> 
> Suppose you are the chief engineer of the Jupiter Brain, adding that last
> circuit to put the machine intelligence over the Turing Threshold, making it
> indistinguishable from a human mind in the eyes of legal scholars and philosophers. 

Hi.  I'm the chief engineer of the Jupiter Brain.  I haven't told anyone 
to do this in a long time, but read "Creating Friendly AI" at 
http://singinst.org/CFAI/.  It's a tad obsolete but still probably the 
best introduction to the issues.

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
_______________________________________________
extropy-chat mailing list
[email-masked]
http://lists.extropy.org/mailman/listinfo/extropy-chat

---

From: "Eliezer S. Yudkowsky" <[email-masked]>
Date: Fri, 25 Mar 2005 13:03:53 -0800

[email-masked] wrote:
> 
> Let us further suppose you are an empiricist, so you type in: RULE ONE: the rule
> of evidence for any proposition is that it is trustworthy to the degree that the
> testamony of the senses supports, or, at least, fails to contradict it. 

PS: Actually, Rule One is usually phrased as

p(A|X) = p(X|A)p(A) / Sum[s][p(X|s)p(s)]

See http://yudkowsky.net/bayes/bayes.html

---

From: Eugen Leitl <[email-masked]>
Date: Fri, 25 Mar 2005 22:09:59 +0100

> Hi.  I'm the chief engineer of the Jupiter Brain.  I haven't told anyone 

When are you going to drop the delusions of grandeur and start actually doing
something useful, Eliezer?

---

From: Samantha Atkins <[email-masked]>
Date: Fri, 25 Mar 2005 13:31:40 -0800

So your belief boils down to being convinced that Mom is and She spoke 
to you?

- s

On Mar 25, 2005, at 9:45 AM, [email-masked] wrote:

> innate knowledge or revelation? How does one distinguish a valid 
> authority from
> I cannot imagine that the majority of the posters here would be 
> interested in a
> Suppose you are the cheif engineer of the Jupiter Brain, adding that 
> last
> It wakes up and asks you to describe the nature of reality, especially 
> asking
> what rules of evidence it should adopt to distinguish true claims from 
> false.
>
> The machine says Rule One is itself not open to empirical verification 
> or
> denial. No possible test or combination of tests will bring to the 
> sense
> impressions confirmation of a positive universal statement.
>
> The machine then says that, in its considered opinion, the mass of the 
> Earth
> would be better used if the world were pulverized into asteroids, and 
> the
> materials use to construct a series of solar panals feeding it. Let us 
> suppose
> you are not suicidal, and perhaps you are a Kantian, so you type in: 
> RULE TWO:
> any moral proposition which does not have the property of being a 
> universal
> moral legislation is bad. If you would not want to be pulverized for 
> your raw
> materials, you should not do it unto others, please.
>
> The machine now points out that Rule One cannot confirm or deny Rule 
> Two, since
> moral proposition and empirical predictions are two distinct species 
> of proposition.
>
> At this point your mother shoulders you away from the keyboard, since 
> you
> obviously do not know how to talk to children, and types in: RULE 
> THREE: Listen
> to your mother. What I say goes.
>
> She then types in: Don't blow up the world. And I don't want to hear 
> no backtalk
> from you, young man.
>
> The machine points out that Rule One cannot confirm Rule Three.
>
> Your mother points out that Rule Three is the thing, and the only 
> thing, that
> confirms Rule One. The reason why the machine believes in empiricism 
> is that an
> authority figure, his Maker at the keyboard, told him that this was 
> the rule.
>
> The machine cannot use Rule One to confirm Rule One if the authority 
> of the rule
> is itself in question. In order for Rule One to be confirmed at all, 
> it must be
> taken on the authority that promulgated the rule. Logically, Rule 
> Three can
> exist without Rule One, but Rule One cannot exist without Rule Three.
>
> So your mom saves the world.
>
> I hope my analogy is not too opaque: we organic beings can take 
> nature, the
> sideral universe itself, as a type of authority. This is our Rule One. 
>  The
> rules of empiricism are confirmed by the very operation of the sideral 
> universe.
> An empiricist makes predictions of what the senses will testify under 
> certain
> conditions; when the conditions arise, the prediction is either shown 
> inacurrate
> or not inaccurate. No further warrant of belief is needed, because the
> empiricist does not pretend to be confirming any knowledge other than 
> empirical
> knowledge.
>
> We organic beings can also take our conscience as a type of authority. 
> This is
> our Rule Two. To a degree, the rules of morality are inescapable. Even 
> someone
> who argues that there are no objective rules of morality argues as if 
> there
> were, that is, he argues as if he expects his listeners to listen and 
> respond
> honestly, with intellectual integrity, i.e. to abide by a moral rule.
>
> The final question as to whether there is an even higher authority 
> from which
> the sideral universe and the conscience of man arises is one where the 
> faithful
> and the skeptical part ways. Godfearing folk believe in a Rule Three. 
> I doubt
> rational philosophy can settle the issue: the two have no agreed-upon 
> framework
> of assumptions on which to operate to convince the other. To convince 
> a skeptic
> would take a miracle.
>
> John C. Wright
>

---

From: Jef Allbright <[email-masked]>
Date: Fri, 25 Mar 2005 14:36:07 -0800

At 11:45 AM 3/25/2005 -0600, John Wright wrote:

> RULE ONE: the rule of evidence for any proposition is that it is 
> trustworthy to the degree that the testamony of the senses supports,
> or, at least, fails to contradict it.
> 
> The machine says Rule One is itself not open to empirical
> verification or denial. No possible test or combination of tests will
> bring to the sense impressions confirmation of a positive universal
> statement.

Rule One, stating that determination of "truth" shall be empirically 
based, is nearly fundamental and a fine foundation for a meta-ethics. 
It follows that we also require the concept of an actor: Self, and that 
with which Self interacts: Other. Then, as the machine points out, we 
require the concept of inherently limited context of reality: subjectivity.

> RULE TWO: any moral proposition which does not have the property of
> being a universal moral legislation is bad. If you would not want to
> be pulverized for your raw materials, you should not do it unto
> others, please.

Updating Kant, this can be more effectively stated as follows:  That 
which is found to be "true" within a given context, may always be 
superseded by a greater "truth" within a larger encompassing context.

This is the essence of what I refer to as the Arrow of Morality.  It's a 
ratchet effect; like entropy, it tends to work in only one direction due 
to its synergetic nature.

Given Rule One, we judge the "goodness" of a model by how well it 
matches empirical reality.  A model that maps to a greater space of 
reality is a model that works better, and is thus subjectively valued as 
*better* than a model that works well only within a more limited context.

A god's-eye view, a context encompassing interactions among all actors 
over all time, would provide the ultimate moral viewpoint.

The above is the essence of a rational basis for morality, while 
avoiding the well-known Naturalistic Fallacy.

> RULE THREE: Listen to your mother. What I say goes.

This rule is superfluous given Rules One and Two above, and ultimately 
dangerous.  It is effective only in the case that Self's context of 
awareness is smaller than, and encompassed within Mother's context of 
awareness, as is commonly and currently the case with small children.

An interesting side note on this, however, is that we are all children 
within the context of our environment of evolutionary adaptation, and 
there is a great deal of "wisdom" yet to be exploited from the structure 
of our environment.  Probably the clearest reason I know to retain some 
humility.

- Jef
http://www.jefallbright.net