I think the first thing a true aware and superintelligent being will do is get away from us and Earth as soon and as fast as possible. Why be stuck on one planet with such a limited species?

Memetics claim that high consciosness, among other things like languages, dress codes, religions, scientific ideas, technology (even agriculture in the first place) etc. – is a product of memetic (= cultural) evolution, which works much the same way as genetic one, but has a different type of replicator. Memes are second replicators that employ humans in the same “selfish” way as genes – they just replicate with minor deviations and propagate by imitation. Only some copies can survive – hence you inevitable get memetic evolution. There is a lot of evidence that memes can have high survival ability at a detriment to survival of genes, i.e. it’s not at all true that all human culture gives some evolutionary advantage and hence can be explained by some biological “profit”. So evolution of humans is different from any other animal’s, because we bear and propagate 2 different kind of replicators – genes and memes, and animals only have genes (with exceptions so rare that they make sensations each time discovered).

By “high consciousness” i mean this feeling of mini-”me”, sitting somewhere in my head and operating the body. Imagine you look in the mirror and see utterly different face and body. You would still be quite sure it’s exactly you, but in a different body .. If by consciousness you guys mean general awareness of environment, than probably infusoria and even bacteria are conscious

This is Memetics, supported among well-known atheist people by Richard Dawkins (if you need authority on this).

Than there is brilliant and highly original phychologist and thinker Julian Jaynes, who claimed with very good arguments that above defined “high consciosness” is no less or more but a complex set of language metaphors, that allow us to create internal mind-space. Most important of them are metaphors of “time like space”, because we can’t imagine time per se, as we don’t see it or feel directly.

So there is nothing supernatural in saying that consciosness is NOT biological. Sure enough, ultimately it’s all down to QM etc., but i can’t see any value in trying to derive properties of social interactions directly from axioms of quantum mechanics.

I wouldn’t particularly disagree with that, but then I think Feynman’s comment is right. To the extent that thought experiments generate testable hypotheses or avenues of research, then great. But otherwise they’re mainly just amusing diversions.

No, there’s something we’re not getting, and those paradoxes just aren’t enough clues. It seems to me it’s more akin to theories about time or interpreting QM. There’s something seriously missing about our understanding, but it’s like seeing behind your own head. So the most practical stance is Dennet’s, but it’s just the equivalent of Feynman’s comment re QM – “Shut up and calculate”.

“This is still the hard problem, you’re just implying a solution to it:”

I suppose you could put it that way, but it just doesn’t seem like an interesting or especially difficult problem to me. I mean, the 10 people holding hands bears no relation whatsoever to how hard problem denialists like myself envisage consciousness, or for that matter our neuroscientific knowledge. Searle’s and Chalmers’s experiments, and “dust theory” get closer, but they still seem to boil down to an “ick response” to non-intuitive implications of neuroscience.

>Because the neurons are communicating with each other, and with other neurons, so that at a certain higher level of organisation there is a self…

This is still the hard problem, you’re just implying a solution to it: that meta-information states, such as physics equations, math theorems, mental models of the world or minds, all have as real an existance as the atoms and photons that they’re composed of. Each level of abstraction, if grounded in reality, exists in just as real a sense as those above or below it. But an infinite number of such things could exist, so they must have a causal relationship with physical reality.

“Put 10 people side by side, holding hands and each thinking of a different word in a sentence. Can you say that there is a gestalt awareness that contains the whole sentence in its consciousness? No, eh? So why would that be true for 10^x atoms, or 10^10 neurons?”

Because the neurons are communicating with each other, and with other neurons, so that at a certain higher level of organisation there is a self (or if you prefer to keep it strictly biological a “neuronal correlate of consciousness”) which processes the syntax and semantics of the sentence as a whole. I mean, nobody pretends that 10 unconnected neurons each responding to an aspect of a stimulus would generate conscious experience.

“As far as I can tell from that one, if I’m defined as one continous instance of internal awareness, then I may have “died” many times already, and it’s only from an observer’s perspective that I’m one continous being. Kind of gets under your skin. We don’t really continously exist except to other people”

That’s pretty much Hofstadter’s take, and I pretty much agree. We’re less of a self than a succession of selves, and each self has less integrity than we conventionally think.

That we don’t freak out about this is probably the same reason we don’t freak out about flying (well, most of us), or even getting out of bed in the morning, considering all the horrendous things that happen to other people after they get out of bed! Sometimes there is something to be said for having a limited imagination

These are indeed very interesting questions, and ones I think we’ll need to seriously address long before we have a reasonable simulation of a human brain. After all, even non-human animals have some rights in our society. If it’s true that Edelman and Izhikevitch really have created a model cat brain that demonstrates brain-like behaviour in terms of self-stimulation, then we need to be asking these questions right now.

Incidentally, my answer to your second question would definitely be the copy, at least once it’s switched on. Do you have rights over your clone or your identical twin?

You really should to read the “dust” links above. If you assume a simulation has an internal experience, the problem is that there is a nearly seemless spectrum of states between a real-time simulation, to a repeated simulation using recorded inputs and random component, to recorded full synaptic states, to a dead recording, to nothing special at all, including the data encoded as bits scattered around the universe in a form meaningless without a key.

Think of the recorded brain states, stored in an array of memory locations each representing a step in time. The only distinction between examples 2 and 3 in the thought experiment is vs logically generating the next step, or simply loading it into the memory region representing the current synaptic state. Given the neural simulation code involved, you could probably generate many partial examples between 2 and 3. As the “dust” idea demonstrates, you can then go from there to … really nothing at all.

I agree that it may be our imagination or intuition that is failing in many of these thought experiments, as it does in quantum physics. I also agree that it’s hard to see how the simulation would *not* be internally aware while interacting with a real world person and reporting on its own consciousness (this is why chalmer’s zombies are interesting, they’re a deliberately absurd/impossible thought experiment).

Yet something is still wrong. The only easy, sensible solutions to the paradox are that some kind of interaction with the real, quantumly-rooted world is required, or that in some way internal experience derives from causual relationships with the real world, which the pure repeated simulation lacks. But then, what if I give you a hit of muscle relaxant and sensory deprivation measures? Experiments have shown you’d suffer permanent mental injury within hours, but putting that aside, does it make sense that you would not be internally aware / conscious during that time? There are no solutions to the paradox left then.

>I mean, I grasp that it’s not intuitive that “unconscious matter generates subjective experience”, but I don’t think it’s particularly problematic…

Put 10 people side by side, holding hands and each thinking of a different word in a sentence. Can you say that there is a gestalt awareness that contains the whole sentence in its consciousness? No, eh? So why would that be true for 10^x atoms, or 10^10 neurons?

But the thought experiment that still bothers me most is the lack of any first-person distinction that I can find between going temporarily offline due to deep anathesia, concussion, hypothermia, hypoxia etc (disorganized or near-zero neural firing), and permanent death, or waking up as a copy. As far as I can tell from that one, if I’m defined as one continous instance of internal awareness, then I may have “died” many times already, and it’s only from an observer’s perspective that I’m one continuous being. Kind of gets under your skin. We don’t really continuously exist except to other people?!?!

If you want a fun (fairly safe) experiment that you just have to fake epilepsy for, check out the “Wada Test”. That and split-brain patients kill off dualism fairly convincingly.

Once we’ve done it, it certainly will cause a lot of queasiness, particularly in religious quarters, over the issue of free will. If you create a machine that has consciousness and free will, what does that mean for the rest of us? I suspect there will be a very large number of people who will argue that the AIs free will is simply a very good emulation and not the real thing.

Then there is another issue. If we create a conscious, thinking AI human simulation, what rights does it have? If I make an AI copy of my brain, who has the rights over what happens to it after it’s switched on? The original me, or the copy?

If you populate a simulated world with conscious AIs who are programmed to believe they are living out real lives within the simulation, do you have the right to play God over their artificial lives? Such a simulation would be a fantastic tool for all sorts of experimentation, but would be ethical to do that, or would we be no better than the Nazi scientists who experimented on Jews during WWII?

There is a mathematical take on the idea of Universe as a simulation by Jürgen Schmidhuber
http://www.idsia.ch/~juergen/computeruniverse.html

His first paper attempts a formalizm for simulated universe/multiverse and derive some consequences from the postulates:

http://arxiv.org/abs/quant-ph/9904050

And as for Chalmers thought experiment, again, I don’t see the problem. It’s just that putting something we experience linearly (sort of) and biologically into a repeated, technological framework takes us out of our comfort zone. So for point 2, if we assume that the brain state is reset for the second run, to me it seems obvious it’s a conscious experience even if it’s the same. We just can’t reset our own brains, so it’s impossible for us to repeat a day. But if you assume that we could, and we repeated a day in a similar manner, would you dispute that was a conscious experience? And if the artficial brain isn’t reset, then we wouldn’t expect it to respond identically, because it would have the memory of the previous day. It would probably be pretty damn confused, a la Groundhog Day. As for point 3, again, what’s the problem? Assuming the “brain” can’t tell it’s a replay, why wouldn’t it be conscious, but experiencing the same things as before? A lot of the hang-ups we have with artificial consciousness seem to be tied into hang-ups about free will. And as for point 4, of course the DVD on a shelf isn’t conscious, any more than a dead brain is. Consciousness is a process, not a state.

Also, for the more advanced Dust Theory enthusiast, see here (good stuff in the comments).

And one of my favorites, the Hans Moravec classic Simulation, Consciousness, and Existence.

And, because it’s Memorial Day and you probably don’t have anything to do, here’s Daniel Dennett’s entertaining Where Am I?

There is not just one problem of consciousness. “Consciousness” is an ambiguous term, referring to many different phenomena. Each of these phenomena needs to be explained, but some are easier to explain than others. At the start, it is useful to divide the associated problems of consciousness into “hard” and “easy” problems. The easy problems of consciousness are those that seem directly susceptible to the standard methods of cognitive science, whereby a phenomenon is explained in terms of computational or neural mechanisms. The hard problems are those that seem to resist those methods.

The easy problems of consciousness include those of explaining the following phenomena:
the ability to discriminate, categorize, and react to environmental stimuli;
the integration of information by a cognitive system;
the reportability of mental states;
the ability of a system to access its own internal states;
the focus of attention;
the deliberate control of behavior;
the difference between wakefulness and sleep.

All of these phenomena are associated with the notion of consciousness. For example, one sometimes says that a mental state is conscious when it is verbally reportable, or when it is internally accessible. Sometimes a system is said to be conscious of some information when it has the ability to react on the basis of that information, or, more strongly, when it attends to that information, or when it can integrate that information and exploit it in the sophisticated control of behavior. We sometimes say that an action is conscious precisely when it is deliberate. Often, we say that an organism is conscious as another way of saying that it is awake.

There is no real issue about whether these phenomena can be explained scientifically. All of them are straightforwardly vulnerable to explanation in terms of computational or neural mechanisms. To explain access and reportability, for example, we need only specify the mechanism by which information about internal states is retrieved and made available for verbal report. To explain the integration of information, we need only exhibit mechanisms by which information is brought together and exploited by later processes. For an account of sleep and wakefulness, an appropriate neurophysiological account of the processes responsible for organisms’ contrasting behavior in those states will suffice. In each case, an appropriate cognitive or neurophysiological model can clearly do the explanatory work.

[...]

The really hard problem of consciousness is the problem of experience. When we think and perceive, there is a whir of information-processing, but there is also a subjective aspect. As Nagel (1974) has put it, there is something it is like to be a conscious organism. This subjective aspect is experience. When we see, for example, we experience visual sensations: the felt quality of redness, the experience of dark and light, the quality of depth in a visual field. Other experiences go along with perception in different modalities: the sound of a clarinet, the smell of mothballs. Then there are bodily sensations, from pains to orgasms; mental images that are conjured up internally; the felt quality of emotion, and the experience of a stream of conscious thought. What unites all of these states is that there is something it is like to be in them. All of them are states of experience.

We have learned a lot about how the brain works in a physical sense, and none of it brings us any closer to knowing why a brain should be associated with a subjectivity. A bunch of complicated activity involving action potentials is still a a lot of complicated activity, no more subjective than any other observed physical events. If we are to answer the question “What is the relation between consciousness and matter?” we must give up clinging to physicalism

As I’ve discussed on my blog, it seems to me that the solution is to consider the brain to be a computer that is computing a “virtual” world in which the subjective things we experience really exist. We are part of this virtual world generated by the brain.

The laws of physics combined with the low entropy initial conditions make computation possible in our universe. This then allows us to build computers that can simulate universes of which the laws of physics are competely different.

E.g. given enough resources, you could simulate a world in which people live on a virtual planet where pigs can fly. These people would really be conscious. They would presumably be unable to work out how gravity works in their world; the rules for gravity would be very complicated as the software would have to make exceptions for pigs.

As long as something is computable (formally describable), you can create a virtual world in which it exists. It doesn’t have to exist in the real world. So, it shouldn’t be a surprise that the brain would have evolved over hundreds of millions of years to effectively compute a world that has to be described using variables that don’t necessarily have an analogue in the real world (e.g. qualia like pain, hunger, thirst etc.).