Activity Feed
#5745·Vincent Brunsch, 13 days agoThe evidential standard seems asymmetric.
You have said that there ultimately cannot be a purely behavioral criterion for consciousness, because what matters is an explanation of the underlying program (#5693). But at the same time, apparently rigid or “blatantly robotic” behavior is allowed to count as evidence against consciousness.
In the other direction, apparently flexible, sophisticated or sentient-like animal behavior does not seem able to count correspondingly in favor of consciousness, because it could always in principle have been produced by a nonconscious program.
If behavior alone cannot establish sentience because we need to understand the underlying processing, I don’t see why apparently robotic behavior should by itself count more readily toward insentience. Unless there is already an independent reason to think phenomenal experience requires universal creativity, the evidential standard seems to favor insentience in advance.
… because it could always in principle have been produced by a nonconscious program.
I don’t think that’s right. There could be animal behavior I’d have serious trouble explaining.
People asked me about this at Conjecture Con. For example, if we took an animal out of the context in which it evolved, and put it in a new context it didn’t evolve for – that’s where animals bug out – but instead of bugging out, it behaved more like a human in an unknown situation and became reflective, actually seemed to understand what was going on, that would at least give me pause.
I can’t be more specific than that because it would depend in part on new knowledge the animal would have to create in that situation, and by definition I can’t predict new knowledge. But I’d know it when I see it.
That may not be completely satisfying, but it shows that it’s simply not true that any animal behavior “could always in principle have been produced by a nonconscious program.”
#5745·Vincent Brunsch, 13 days agoThe evidential standard seems asymmetric.
You have said that there ultimately cannot be a purely behavioral criterion for consciousness, because what matters is an explanation of the underlying program (#5693). But at the same time, apparently rigid or “blatantly robotic” behavior is allowed to count as evidence against consciousness.
In the other direction, apparently flexible, sophisticated or sentient-like animal behavior does not seem able to count correspondingly in favor of consciousness, because it could always in principle have been produced by a nonconscious program.
If behavior alone cannot establish sentience because we need to understand the underlying processing, I don’t see why apparently robotic behavior should by itself count more readily toward insentience. Unless there is already an independent reason to think phenomenal experience requires universal creativity, the evidential standard seems to favor insentience in advance.
But at the same time, apparently rigid or “blatantly robotic” behavior is allowed to count as evidence against consciousness.
I think animals are so blatantly mindless and stupid, like so far off the charts, that it counts for something. I’d bet money that you use the same standard for a Tesla bot or a Roomba, even though neither of us has a working theory of consciousness.
#5744·Vincent Brunsch, 13 days agoI agree that this is not “learning” in the special Deutschian sense of universal knowledge creation. I was using “learning” in the ordinary biological sense of experience-dependent changes in processing and behavior.
In fact, the passage you link makes that distinction explicitly: Deutsch allows that the robot’s useful change can be called learning, while denying that it constitutes genuine creation of new knowledge.
That distinction is actually part of my point. I’m not claiming continuity in universal creativity between humans and other animals. I’m pointing to continuity in the older, non-universal capacities (such as sensory, nociceptive, affective and ordinary learning processes). Whether those processes can support phenomenal experience is precisely what is under discussion.
Ok but whether systems unrelated to learning in the Deutschian sense have continuity doesn’t seem to matter then.
The evolution of feathers has continuity in birds. So what? How does that matter for consciousness?
#5743·Vincent Brunsch, 13 days agoI also wonder whether your note here should be a criticism rather than a comment. As I understand Veritula, an outstanding criticism makes the parent idea problematic and therefore not rationally adoptable, whereas comments are neutral.
A suggestion that an idea could be phrased more simply seems like a useful comment, but unless the wording actually makes the content of the claim ambiguous or substantively defective, I don’t see why it should count as a reason to reject the idea itself.
Criticisms target ideas as written. That includes grammar, typos, complexity/simplicity, etc.
#5742·Vincent Brunsch, 13 days agoI initially wondered whether you were right that I could simplify this, but I think the distinction is necessary here. Writing “they … feel pain” would already assume part of the conclusion under dispute: that animal nociceptive processing is accompanied by phenomenal pain.
By “sensory, affective, nociceptive and learning mechanisms” I was trying to refer to processes we can identify without assuming that they involve conscious experience. The wording is less simple, but in this case I think the precision actually matters.
Precision isn’t the issue here. Simple language can be precise. Read Popper and Feynman on expressing yourself so your readers understand you and your ideas get easier to criticize.
#5741·Vincent Brunsch, 13 days agoI agree of course that consciousness, if computational, need not depend on neurons as a particular material substrate. But that is different from saying that the organization and activity implemented by those neurons tell us nothing.
The article you link makes this distinction itself: substrate independence means that most details of the substrate do not matter; what matters is the relevant pattern or structure of information processing. Studying homologous neural systems can give us evidence about exactly that organization and processing.
So my claim is not “they have similar neurons, therefore they have similar experiences.” It is that there is evolutionary continuity not only in the neural substrate, but also in how systems involved in sensory, nociceptive and affective processing are organized, what signals they process, and how they interact with other systems.
That seems evidentially relevant to what information processing is occurring. Substrate independence does not make that evidence irrelevant.
There’s also continuity in how MacBooks process information over the years, and how their hardware evolved, including ‘sensory processing’ like their cameras and microphones, yet none of them are sentient/conscious, and if one became conscious tomorrow, it would have nothing to do with its hardware and everything to do with its software.
#5740·Vincent Brunsch, 13 days agoYes, it’s a guess. So the leap to universal creativity does not itself imply that phenomenal experience appeared in the same leap.
I proposed an alternative possibility (#5739): older, shared systems may already have supported phenomenal experience before universal creativity emerged. I’m not claiming that this alternative is established either.
I’m not sure “no other explanation is known” is quite right. For example, IIT offers a very different theory under which consciousness depends on a system’s intrinsic causal organization. My conjecture that phenomenal experience could predate universal creativity is at least compatible with that kind of account.
So it seems to me that we currently have competing conjectures here, rather than one known explanation and no alternative.
I’m not claiming that this alternative is established either.
This is again the wrong epistemology.
So it seems to me that we currently have competing conjectures here, rather than one known explanation and no alternative.
It only seems that way because of the wrong epistemology.
#5739·Vincent Brunsch, 13 days agoI agree that shared evolutionary history by itself would not establish that the same “software” is running. But that wasn’t my claim.
My point is that humans inherited from earlier animals many of the same organized systems involved in things like sensory processing, nociception, affect and learning. The emergence of universal creativity could have changed the overall cognitive architecture dramatically without implying that all of those older systems acquired their phenomenal properties only at that same point.
So the laptop analogy shows that common ancestry does not entail identical software, which I agree with. What I don’t think it shows is that the continuity of these particular inherited systems is evidentially irrelevant to whether phenomenal experience may already have been present before universality.
What I don’t think it shows is that the continuity of these particular inherited systems is evidentially irrelevant to whether phenomenal experience may already have been present before universality.
Please don’t write like you’re trying to impress a teacher. Use simple language anyone can understand. If you’re using AI to write or help write your ideas, you must tell people by checking the checkbox in the form.
#5739·Vincent Brunsch, 13 days agoI agree that shared evolutionary history by itself would not establish that the same “software” is running. But that wasn’t my claim.
My point is that humans inherited from earlier animals many of the same organized systems involved in things like sensory processing, nociception, affect and learning. The emergence of universal creativity could have changed the overall cognitive architecture dramatically without implying that all of those older systems acquired their phenomenal properties only at that same point.
So the laptop analogy shows that common ancestry does not entail identical software, which I agree with. What I don’t think it shows is that the continuity of these particular inherited systems is evidentially irrelevant to whether phenomenal experience may already have been present before universality.
What I don’t think it shows is that the continuity of these particular inherited systems is evidentially irrelevant to whether phenomenal experience may already have been present before universality.
It’s the software that would give rise to consciousness because computation is substrate-independent. So yes, I think hardware continuity is irrelevant (unless there’s a switch from non-universal computer to universal or vice versa).
#5738·Vincent Brunsch, 13 days agoBy “phenomenal experience” I simply mean there being something it is like to see red, feel pain, fear, etc. Your claim that animals are insentient is itself a claim that these experiences are absent.
Whether such experience can occur without universal/critical cognition is precisely what is under dispute. So saying that these distinctions may not exist, because ultimately there is one kind of information processing that gives rise to consciousness, seems to assume the package-deal hypothesis rather than provide an argument for it.
If there is an independent reason why phenomenal experience cannot occur without universal creativity, that would address the criticism. But I don’t think collapsing the distinction itself does.
… seems to assume the package-deal hypothesis rather than provide an argument for it.
It sounds like you’re using a non-Popperian epistemology, with positive reasons to adopt some stance.
I don’t think that’s right, for the reasons Popper explained. I don’t make arguments for any stance. I ask if anyone finds anything wrong with it. I keep advocating it until someone does.
#5737·Vincent Brunsch, 13 days agoI don’t think the movie/video-game analogy captures what I find interesting about dreaming here, because it seems to put phenomenal awareness, meta-awareness and control on a single scale of “critical ability”.
These capacities can dissociate: In an ordinary non-lucid dream there can already be vivid phenomenal experience without awareness that one is dreaming. Lucidity adds that meta-awareness, but does not necessarily add control (lucid dreams can themselves occur with or without substantial control over what happens).
So rather than a simple progression from passive/uncritical to active/critical, dreaming seems to give us different combinations of phenomenal experience, meta-awareness and control.
That is precisely why I think it provides some evidence that phenomenal experience is not tightly bound to the other capacities.
That is precisely why I think it provides some evidence that phenomenal experience is not tightly bound to the other capacities.
If that is so, please give example source code that has “phenomenal experience” but no consciousness.
#5736·Vincent Brunsch, 13 days agoI think “awareness” is doing two different jobs here. Of course we are consciously experiencing an ordinary dream (seeing, feeling, fearing, etc.). That is a premise of my criticism.
What is characteristically absent in a non-lucid dream is awareness that one is dreaming, together with much of the associated reality monitoring and critical reflection. Becoming aware that one is dreaming is precisely what distinguishes lucid dreaming.
So saying “we’re aware of them” does not seem to address the distinction I was making between phenomenal experience and reflective/metacognitive awareness.
… the distinction I was making between phenomenal experience and reflective/metacognitive awareness.
As Deutsch likes to emphasize, there’s a G in AGI for a reason. No distinction need be made here.
#5736·Vincent Brunsch, 13 days agoI think “awareness” is doing two different jobs here. Of course we are consciously experiencing an ordinary dream (seeing, feeling, fearing, etc.). That is a premise of my criticism.
What is characteristically absent in a non-lucid dream is awareness that one is dreaming, together with much of the associated reality monitoring and critical reflection. Becoming aware that one is dreaming is precisely what distinguishes lucid dreaming.
So saying “we’re aware of them” does not seem to address the distinction I was making between phenomenal experience and reflective/metacognitive awareness.
What is characteristically absent in a non-lucid dream is awareness that one is dreaming, together with much of the associated reality monitoring and critical reflection.
This sounds like you’re agreeing with me – absence of awareness = absence of critical reflection – yet it’s marked as a criticism.
#5736·Vincent Brunsch, 13 days agoI think “awareness” is doing two different jobs here. Of course we are consciously experiencing an ordinary dream (seeing, feeling, fearing, etc.). That is a premise of my criticism.
What is characteristically absent in a non-lucid dream is awareness that one is dreaming, together with much of the associated reality monitoring and critical reflection. Becoming aware that one is dreaming is precisely what distinguishes lucid dreaming.
So saying “we’re aware of them” does not seem to address the distinction I was making between phenomenal experience and reflective/metacognitive awareness.
OK but there are many things we do while awake that we aren’t aware of either. People aren’t aware always of their accents or ticks or fidgeting. I don’t understand why that matters.
#5735·Vincent Brunsch, 13 days agoI agree that behavior which is blatantly robotic can be evidence that the behavior itself does not involve universal creativity, questioning or error correction. But I don't see how that gets us to the stronger conclusion that it “has nothing to do with consciousness.”
A system could execute a rigid, preprogrammed behavior while still having phenomenal experiences such as seeing color or feeling pain. The robotic character of the behavior seems relevant to whether the system is exercising universal creativity, but not by itself to whether anything is being experienced.
To infer insentience from the robotic behavior, we would still need the separate premise that phenomenal experience cannot accompany non-universal information processing, which is the point under dispute.
To infer insentience from the robotic behavior, we would still need the separate premise that phenomenal experience cannot accompany non-universal information processing, which is the point under dispute.
Same goes for inferring sentience from robotic behavior; for the claim that phenomenal experience can accompany non-universal information processing.
Hence my suggestion that conscious information processing is critical.
Humans working at an assembly line can be ‘robotic’ in the sense you describe. That’s a fair point. But then they go home and don’t act robotically.
Sleepwalkers are another example.
The claim is that animals always act robotically (outside of just plain random or buggy behavior). They can’t do otherwise. The claim is not that humans never act robotically. They do, sometimes, but they can be critical and change course.
Humans working at an assembly line can be ‘robotic’ in the sense you describe. That’s a fair point. But then they go home and don’t act robotically.
Sleepwalkers are another example: they act robotically/uncritically while sleepwalking, but then they wake up and stop.
The claim is that animals always act robotically (outside of just plain random or buggy behavior). They can’t do otherwise. The claim is not that humans never act robotically. They do, sometimes, but they can be critical and change course.
#5735·Vincent Brunsch, 13 days agoI agree that behavior which is blatantly robotic can be evidence that the behavior itself does not involve universal creativity, questioning or error correction. But I don't see how that gets us to the stronger conclusion that it “has nothing to do with consciousness.”
A system could execute a rigid, preprogrammed behavior while still having phenomenal experiences such as seeing color or feeling pain. The robotic character of the behavior seems relevant to whether the system is exercising universal creativity, but not by itself to whether anything is being experienced.
To infer insentience from the robotic behavior, we would still need the separate premise that phenomenal experience cannot accompany non-universal information processing, which is the point under dispute.
Humans working at an assembly line can be ‘robotic’ in the sense you describe. That’s a fair point. But then they go home and don’t act robotically.
Sleepwalkers are another example.
The claim is that animals always act robotically (outside of just plain random or buggy behavior). They can’t do otherwise. The claim is not that humans never act robotically. They do, sometimes, but they can be critical and change course.
#5734·Vincent Brunsch, 13 days agoSuppose we eventually can write complete generative programs for both a crow and a human. You say that the human program “would result in consciousness.”
I agree that this follows from your theory if universal creativity entails consciousness. But then I don’t think it can also serve as an independent argument for that connection. The possibility of a universally creative philosophical zombie is ruled out by your theory, rather than independently by the fact that the program is universally creative.
Likewise, your theory predicts that the crow program would be unconscious if it remains non-universal. Competing theories of consciousness could make different predictions about the same two programs. For example, IIT explicitly allows functionally equivalent systems to differ in consciousness depending on their intrinsic causal organization. I’m not assuming IIT is correct. It just illustrates that knowing what the two programs can do would not by itself decide which theory of consciousness is correct.
So I think the simulation comparison inherits the disagreement about which theory of consciousness is correct; it does not by itself resolve the link between universal creativity and phenomenal experience.
Rival predictions don’t keep a question open. By that standard, the question of whether rocks or thermostats are conscious stays open too. IIT gives some consciousness even to a photodiode, and panpsychism gives it to everything. That rival theories predict different things doesn’t leave a question undecided, or no question would ever be decided. What settles it is which rivals survive criticism.
#5734·Vincent Brunsch, 13 days agoSuppose we eventually can write complete generative programs for both a crow and a human. You say that the human program “would result in consciousness.”
I agree that this follows from your theory if universal creativity entails consciousness. But then I don’t think it can also serve as an independent argument for that connection. The possibility of a universally creative philosophical zombie is ruled out by your theory, rather than independently by the fact that the program is universally creative.
Likewise, your theory predicts that the crow program would be unconscious if it remains non-universal. Competing theories of consciousness could make different predictions about the same two programs. For example, IIT explicitly allows functionally equivalent systems to differ in consciousness depending on their intrinsic causal organization. I’m not assuming IIT is correct. It just illustrates that knowing what the two programs can do would not by itself decide which theory of consciousness is correct.
So I think the simulation comparison inherits the disagreement about which theory of consciousness is correct; it does not by itself resolve the link between universal creativity and phenomenal experience.
It just illustrates that knowing what the two programs can do would not by itself decide which theory of consciousness is correct.
There would need to be something different about a program that gives rise to consciousness. We just haven’t seen any such program because we don’t know how to write one. But I do think the difference would be apparent once we see the source code.
#5734·Vincent Brunsch, 13 days agoSuppose we eventually can write complete generative programs for both a crow and a human. You say that the human program “would result in consciousness.”
I agree that this follows from your theory if universal creativity entails consciousness. But then I don’t think it can also serve as an independent argument for that connection. The possibility of a universally creative philosophical zombie is ruled out by your theory, rather than independently by the fact that the program is universally creative.
Likewise, your theory predicts that the crow program would be unconscious if it remains non-universal. Competing theories of consciousness could make different predictions about the same two programs. For example, IIT explicitly allows functionally equivalent systems to differ in consciousness depending on their intrinsic causal organization. I’m not assuming IIT is correct. It just illustrates that knowing what the two programs can do would not by itself decide which theory of consciousness is correct.
So I think the simulation comparison inherits the disagreement about which theory of consciousness is correct; it does not by itself resolve the link between universal creativity and phenomenal experience.
For example, IIT explicitly allows functionally equivalent systems to differ in consciousness depending on their intrinsic causal organization.
I don’t see how that can be possible. Surely the same source code would arise in consciousness every time?
#5733·Vincent Brunsch, 13 days agoI understand “we can write the programs and simulate animals” to mean that we can write an underlying program capable of generating an animal’s full behavioral and cognitive repertoire across novel circumstances. If you only mean that we can reproduce particular behaviors, then I don’t see the asymmetry, since we can reproduce particular human behaviors computationally too.
But I think we should be careful with the stronger claim. As far as I know, we cannot currently write such a complete generative program for animals such as crows or apes either. Indeed, even for vastly simpler animals such as the fruit fly or C. elegans, despite having mapped its complete connectome, our executable models still reproduce only parts of its behavior rather than the organism’s full repertoire.
… our executable models still reproduce only parts of its behavior rather than the organism’s full repertoire.
Maybe so, still I have seen nothing any animal does that could not be explained through the uncritical/mindless execution of existing knowledge or plain bugs.
#5733·Vincent Brunsch, 13 days agoI understand “we can write the programs and simulate animals” to mean that we can write an underlying program capable of generating an animal’s full behavioral and cognitive repertoire across novel circumstances. If you only mean that we can reproduce particular behaviors, then I don’t see the asymmetry, since we can reproduce particular human behaviors computationally too.
But I think we should be careful with the stronger claim. As far as I know, we cannot currently write such a complete generative program for animals such as crows or apes either. Indeed, even for vastly simpler animals such as the fruit fly or C. elegans, despite having mapped its complete connectome, our executable models still reproduce only parts of its behavior rather than the organism’s full repertoire.
… since we can reproduce particular human behaviors computationally too
Only for very narrow tasks, and even then there’s no guarantee humans won’t deviate from it, whereas for animals there basically is. Humans aren’t predictable like that but animals are.
Quote Checker now supports MLA and APA style guides, helping you quote more accurately:
https://quote-checker.com
Quote Checker now supports MLA, APA, and Chicago style guides, helping you quote more accurately:
https://quote-checker.com
Quote Checker now supports MLA and APA style guides, helping you quote more accurately:
https://quote-checker.com
What If You Could Hear Your Code?
https://blog.dennishackethal.com/posts/what-if-you-could-hear-your-code