I think the first thing a true aware and superintelligent being will do is get away from us and Earth as soon and as fast as possible. Why be stuck on one planet with such a limited species?

Memetics claim that high consciosness, among other things like languages, dress codes, religions, scientific ideas, technology (even agriculture in the first place) etc. – is a product of memetic (= cultural) evolution, which works much the same way as genetic one, but has a different type of replicator. Memes are second replicators that employ humans in the same “selfish” way as genes – they just replicate with minor deviations and propagate by imitation. Only some copies can survive – hence you inevitable get memetic evolution. There is a lot of evidence that memes can have high survival ability at a detriment to survival of genes, i.e. it’s not at all true that all human culture gives some evolutionary advantage and hence can be explained by some biological “profit”. So evolution of humans is different from any other animal’s, because we bear and propagate 2 different kind of replicators – genes and memes, and animals only have genes (with exceptions so rare that they make sensations each time discovered).

By “high consciousness” i mean this feeling of mini-”me”, sitting somewhere in my head and operating the body. Imagine you look in the mirror and see utterly different face and body. You would still be quite sure it’s exactly you, but in a different body .. If by consciousness you guys mean general awareness of environment, than probably infusoria and even bacteria are conscious

This is Memetics, supported among well-known atheist people by Richard Dawkins (if you need authority on this).

Than there is brilliant and highly original phychologist and thinker Julian Jaynes, who claimed with very good arguments that above defined “high consciosness” is no less or more but a complex set of language metaphors, that allow us to create internal mind-space. Most important of them are metaphors of “time like space”, because we can’t imagine time per se, as we don’t see it or feel directly.

So there is nothing supernatural in saying that consciosness is NOT biological. Sure enough, ultimately it’s all down to QM etc., but i can’t see any value in trying to derive properties of social interactions directly from axioms of quantum mechanics.

I wouldn’t particularly disagree with that, but then I think Feynman’s comment is right. To the extent that thought experiments generate testable hypotheses or avenues of research, then great. But otherwise they’re mainly just amusing diversions.

No, there’s something we’re not getting, and those paradoxes just aren’t enough clues. It seems to me it’s more akin to theories about time or interpreting QM. There’s something seriously missing about our understanding, but it’s like seeing behind your own head. So the most practical stance is Dennet’s, but it’s just the equivalent of Feynman’s comment re QM – “Shut up and calculate”.

“This is still the hard problem, you’re just implying a solution to it:”

I suppose you could put it that way, but it just doesn’t seem like an interesting or especially difficult problem to me. I mean, the 10 people holding hands bears no relation whatsoever to how hard problem denialists like myself envisage consciousness, or for that matter our neuroscientific knowledge. Searle’s and Chalmers’s experiments, and “dust theory” get closer, but they still seem to boil down to an “ick response” to non-intuitive implications of neuroscience.

>Because the neurons are communicating with each other, and with other neurons, so that at a certain higher level of organisation there is a self…

This is still the hard problem, you’re just implying a solution to it: that meta-information states, such as physics equations, math theorems, mental models of the world or minds, all have as real an existance as the atoms and photons that they’re composed of. Each level of abstraction, if grounded in reality, exists in just as real a sense as those above or below it. But an infinite number of such things could exist, so they must have a causal relationship with physical reality.

“Put 10 people side by side, holding hands and each thinking of a different word in a sentence. Can you say that there is a gestalt awareness that contains the whole sentence in its consciousness? No, eh? So why would that be true for 10^x atoms, or 10^10 neurons?”

Because the neurons are communicating with each other, and with other neurons, so that at a certain higher level of organisation there is a self (or if you prefer to keep it strictly biological a “neuronal correlate of consciousness”) which processes the syntax and semantics of the sentence as a whole. I mean, nobody pretends that 10 unconnected neurons each responding to an aspect of a stimulus would generate conscious experience.

“As far as I can tell from that one, if I’m defined as one continous instance of internal awareness, then I may have “died” many times already, and it’s only from an observer’s perspective that I’m one continous being. Kind of gets under your skin. We don’t really continously exist except to other people”

That’s pretty much Hofstadter’s take, and I pretty much agree. We’re less of a self than a succession of selves, and each self has less integrity than we conventionally think.

That we don’t freak out about this is probably the same reason we don’t freak out about flying (well, most of us), or even getting out of bed in the morning, considering all the horrendous things that happen to other people after they get out of bed! Sometimes there is something to be said for having a limited imagination