Vincent Brunsch

That all makes sense to me, thanks. But there is a cost: my previous badges were removed which means that my contributions would have stayed limited to 3 per day if I understand the rules correctly. This should change of course, now, that I do not use AI for responses, so I should be fine in the log run, but my friend, who would then disclose that he uses AI for all his discussions, would never be able to submit more than 3 ideas per day, and would also never be able to join more than one discussion per week because none of his responses would count towards earning the necessary badges https://veritula.com/badges.

My criticism goes deeper though: I really think that there is a case to make that for the goal you describe, to find a bit of truth (or maybe just the current best explanation), it is actually better to use AI, simply because we can spend less time correcting bad faith, misunderstandings, unclear language etc. (Also, one might just say "well, so it takes a bit more time, no problem" - but it's more complicated, because bad faith is often provoking emotions etc which often ends debates before finding a bit more truth)
In our sample discussion about animal consciousness, if I would correct for the "too long" and "not simple enough", I think it would be helpful. But of course, now that I know that you don't appreciate it and since you are currently the only one responding, I will gladly follow your wish not to use it.

I guess, the point I'm trying to make is really that it should not be coming with any form of punishment (like that this does not count for badges) IF the AI assist is clearly done in good faith.

I’m not using AI for this response and will not do so moving forward on Veritula to respect the rules and what seems to be the sentiment behind these rules (unless they change). My criticism is made in good faith and I will try to be especially careful not to make an unwanted mistake in this regard, because usually I would ask AI to check my wording for that.

I’ve been told by someone here that they preferred to talk to a person, not to an AI. This argument is understandable when there is no effort or no thinking involved by the human who uses the AI, or even worse, when the argument is not even looked at and just fed into an LLM to produce some counter, which is disrespectful, does not honor the time, effort and good faith of the other, and why the idea of a meat proxy and the criticism regarding that behavior makes sense. However, in my opinion, AI-assisted can mean many different things, some of which should be encouraged, not restricted or even punished.

Here are two examples that come to mind:

1. My English is not great but the English from a Spanish friend of mine is considerably worse, and I wonder if I should tell him about this platform, because he would usually use AI to polish his English. Instead, of course, he could use Spanish and then let it be translated. I understand this would probably be possible under the current rules but I’m not entirely sure. If it is, would it also be allowed for him to use a tool that makes it sound not like a literal translation, if he still understands afterwards what is being said? An AI assist here seems helpful for everyone involved.

2. All my contributions on this platform before this post have been AI-assisted in part exactly because I wanted to be as respectful of my counterpart’s time and effort (and those of any observers) as possible: I am aware that we sometimes can misunderstand the other’s position or use rhetorical moves without noticing it. So I would send the conversation we have had so far to the AI, share my thoughts on how I understand it and want to proceed, ask it to steelman the other side’s view, and see if its response would help me. It would then be a back and forth where I would try to figure out for myself if I should really change my mind on this or, if not, how exactly I want to respond to the steelmanned version to make my point clear. Now, as a result here I got the criticism that the language should be simpler and the arguments shorter, which is great feedback! I would incorporate that for the future, and any other criticism that I agree would make it better for the discussion. This would then be the current best version I have to offer, but yes, it’s with the use of an AI tool instead of me going out to talk with some philosopher friends about the latest criticism, taking notes, and then have some other friend check my final version for rhetorical moves, clarity, simplicity and shortness before posting it here.

In general, one question in my mind is how future proof this kind of rule is - if we merge more with the tools around us, if they are more integrated in us, would it still not be wanted to use them if done in good faith?

Yes, I think you misunderstood. I've been referring to continuity in the organization of information processing, not the physical substrate. Evolutionary continuity is relevant because humans and other animals share many mechanisms and processing structures, some of which may be relevant to sentience. That doesn't imply that evolution couldn't also produce fundamentally new cognitive capacities.

I think you're still missing the distinction I've made. Initially, you invoked substrate independence as though it contradicted my argument, but I never argued that the physical substrate was what mattered.

As Tegmark explains in the article you linked, it's the organization of information processing that matters, not the material implementing it. And that's exactly what I've been talking about: humans and other animals share evolved mechanisms with significant continuity in how information processing is organized.

Your MacBook analogy doesn't address that. In fact, if a future MacBook became sentient, it would be very interesting to investigate its predecessors (probably) ONLY if they shared a sufficiently similar organization of information processing. If, instead, sentience arose through a fundamentally different program, those predecessors would be much less plausible candidates for sentience.

So when you conclude that it would be about software rather than hardware, you're arguing against a claim I never made.

I think this leaves my central criticism unresolved: Your hypothetical "I'd know it when I see it" seems to test whether an animal possesses universal creativity, whereas my question is whether it can have subjective experiences without possessing that capacity.

Even if an animal never demonstrated the kind of reflective understanding you describe, why would that count as evidence that it cannot feel pain, fear, or anything else?

This seems to depend on the very connection between universal creativity and sentience that is under dispute. What is the independent reason for thinking that noncreative information processing cannot support phenomenal experience?

And I find your sleepwalking example particularly interesting in this regard, because it seems to support the possibility that these capacities don't need to go together. In the video, you yourself point out that the woman can recognize objects and taste, including recognizing that rice cakes are dry, even while asleep.

When she wakes up, however, what seems to change is her capacity for reflective awareness and understanding.

The evidential standard seems asymmetric.

You have said that there ultimately cannot be a purely behavioral criterion for consciousness, because what matters is an explanation of the underlying program (#5693). But at the same time, apparently rigid or “blatantly robotic” behavior is allowed to count as evidence against consciousness.

In the other direction, apparently flexible, sophisticated or sentient-like animal behavior does not seem able to count correspondingly in favor of consciousness, because it could always in principle have been produced by a nonconscious program.

If behavior alone cannot establish sentience because we need to understand the underlying processing, I don’t see why apparently robotic behavior should by itself count more readily toward insentience. Unless there is already an independent reason to think phenomenal experience requires universal creativity, the evidential standard seems to favor insentience in advance.

I agree that this is not “learning” in the special Deutschian sense of universal knowledge creation. I was using “learning” in the ordinary biological sense of experience-dependent changes in processing and behavior.

In fact, the passage you link makes that distinction explicitly: Deutsch allows that the robot’s useful change can be called learning, while denying that it constitutes genuine creation of new knowledge.

That distinction is actually part of my point. I’m not claiming continuity in universal creativity between humans and other animals. I’m pointing to continuity in the older, non-universal capacities (such as sensory, nociceptive, affective and ordinary learning processes). Whether those processes can support phenomenal experience is precisely what is under discussion.

I also wonder whether your note here should be a criticism rather than a comment. As I understand Veritula, an outstanding criticism makes the parent idea problematic and therefore not rationally adoptable, whereas comments are neutral.

A suggestion that an idea could be phrased more simply seems like a useful comment, but unless the wording actually makes the content of the claim ambiguous or substantively defective, I don’t see why it should count as a reason to reject the idea itself.

I initially wondered whether you were right that I could simplify this, but I think the distinction is necessary here. Writing “they … feel pain” would already assume part of the conclusion under dispute: that animal nociceptive processing is accompanied by phenomenal pain.

By “sensory, affective, nociceptive and learning mechanisms” I was trying to refer to processes we can identify without assuming that they involve conscious experience. The wording is less simple, but in this case I think the precision actually matters.

I agree of course that consciousness, if computational, need not depend on neurons as a particular material substrate. But that is different from saying that the organization and activity implemented by those neurons tell us nothing.

The article you link makes this distinction itself: substrate independence means that most details of the substrate do not matter; what matters is the relevant pattern or structure of information processing. Studying homologous neural systems can give us evidence about exactly that organization and processing.

So my claim is not “they have similar neurons, therefore they have similar experiences.” It is that there is evolutionary continuity not only in the neural substrate, but also in how systems involved in sensory, nociceptive and affective processing are organized, what signals they process, and how they interact with other systems.

That seems evidentially relevant to what information processing is occurring. Substrate independence does not make that evidence irrelevant.

Yes, it’s a guess. So the leap to universal creativity does not itself imply that phenomenal experience appeared in the same leap.

I proposed an alternative possibility (#5739): older, shared systems may already have supported phenomenal experience before universal creativity emerged. I’m not claiming that this alternative is established either.

I’m not sure “no other explanation is known” is quite right. For example, IIT offers a very different theory under which consciousness depends on a system’s intrinsic causal organization. My conjecture that phenomenal experience could predate universal creativity is at least compatible with that kind of account.

So it seems to me that we currently have competing conjectures here, rather than one known explanation and no alternative.

I agree that shared evolutionary history by itself would not establish that the same “software” is running. But that wasn’t my claim.

My point is that humans inherited from earlier animals many of the same organized systems involved in things like sensory processing, nociception, affect and learning. The emergence of universal creativity could have changed the overall cognitive architecture dramatically without implying that all of those older systems acquired their phenomenal properties only at that same point.

So the laptop analogy shows that common ancestry does not entail identical software, which I agree with. What I don’t think it shows is that the continuity of these particular inherited systems is evidentially irrelevant to whether phenomenal experience may already have been present before universality.

By “phenomenal experience” I simply mean there being something it is like to see red, feel pain, fear, etc. Your claim that animals are insentient is itself a claim that these experiences are absent.

Whether such experience can occur without universal/critical cognition is precisely what is under dispute. So saying that these distinctions may not exist, because ultimately there is one kind of information processing that gives rise to consciousness, seems to assume the package-deal hypothesis rather than provide an argument for it.

If there is an independent reason why phenomenal experience cannot occur without universal creativity, that would address the criticism. But I don’t think collapsing the distinction itself does.

I don’t think the movie/video-game analogy captures what I find interesting about dreaming here, because it seems to put phenomenal awareness, meta-awareness and control on a single scale of “critical ability”.

These capacities can dissociate: In an ordinary non-lucid dream there can already be vivid phenomenal experience without awareness that one is dreaming. Lucidity adds that meta-awareness, but does not necessarily add control (lucid dreams can themselves occur with or without substantial control over what happens).

So rather than a simple progression from passive/uncritical to active/critical, dreaming seems to give us different combinations of phenomenal experience, meta-awareness and control.

That is precisely why I think it provides some evidence that phenomenal experience is not tightly bound to the other capacities.

I think “awareness” is doing two different jobs here. Of course we are consciously experiencing an ordinary dream (seeing, feeling, fearing, etc.). That is a premise of my criticism.

What is characteristically absent in a non-lucid dream is awareness that one is dreaming, together with much of the associated reality monitoring and critical reflection. Becoming aware that one is dreaming is precisely what distinguishes lucid dreaming.

So saying “we’re aware of them” does not seem to address the distinction I was making between phenomenal experience and reflective/metacognitive awareness.

I agree that behavior which is blatantly robotic can be evidence that the behavior itself does not involve universal creativity, questioning or error correction. But I don't see how that gets us to the stronger conclusion that it “has nothing to do with consciousness.”

A system could execute a rigid, preprogrammed behavior while still having phenomenal experiences such as seeing color or feeling pain. The robotic character of the behavior seems relevant to whether the system is exercising universal creativity, but not by itself to whether anything is being experienced.

To infer insentience from the robotic behavior, we would still need the separate premise that phenomenal experience cannot accompany non-universal information processing, which is the point under dispute.

Suppose we eventually can write complete generative programs for both a crow and a human. You say that the human program “would result in consciousness.”

I agree that this follows from your theory if universal creativity entails consciousness. But then I don’t think it can also serve as an independent argument for that connection. The possibility of a universally creative philosophical zombie is ruled out by your theory, rather than independently by the fact that the program is universally creative.

Likewise, your theory predicts that the crow program would be unconscious if it remains non-universal. Competing theories of consciousness could make different predictions about the same two programs. For example, IIT explicitly allows functionally equivalent systems to differ in consciousness depending on their intrinsic causal organization. I’m not assuming IIT is correct. It just illustrates that knowing what the two programs can do would not by itself decide which theory of consciousness is correct.

So I think the simulation comparison inherits the disagreement about which theory of consciousness is correct; it does not by itself resolve the link between universal creativity and phenomenal experience.

I understand “we can write the programs and simulate animals” to mean that we can write an underlying program capable of generating an animal’s full behavioral and cognitive repertoire across novel circumstances. If you only mean that we can reproduce particular behaviors, then I don’t see the asymmetry, since we can reproduce particular human behaviors computationally too.

But I think we should be careful with the stronger claim. As far as I know, we cannot currently write such a complete generative program for animals such as crows or apes either. Indeed, even for vastly simpler animals such as the fruit fly or C. elegans, despite having mapped its complete connectome, our executable models still reproduce only parts of its behavior rather than the organism’s full repertoire.

In the reasoning you link in support of animal insentience, you argue that similarities between human and animal nervous systems do not tell us whether the same conscious “software” is running. But humans and other mammals are not analogous to two identical computers on which entirely unrelated programs happen to have been installed. The relevant systems have a shared evolutionary history, with substantial continuity in sensory, affective, nociceptive and learning mechanisms.

A change in that inherited system may indeed have produced the qualitative leap to universal creativity. Since universality is necessarily all-or-nothing in the relevant sense, that leap is unsurprising. But it does not follow that every other aspect of mental life (including pain or other phenomenal experience) must have appeared in the same leap, or even require a comparable leap at all.

Neural continuity therefore seems evidentially relevant even if it cannot by itself prove sentience. The observation that universal creativity is uniquely human still leaves open whether older, shared systems already supported phenomenal experience.

If vivid phenomenal experience persists while the relevant critical capacities are greatly diminished (and some forms of meta-awareness appear to be absent altogether in ordinary dreams) that seems at least evidence that phenomenal experience does not closely depend on the degree to which those capacities are active. Lucid dreaming strengthens this: reflective/metacognitive capacities can return within an experience that was already phenomenal.

I have seen no evidence of any animal behavior that could not have been preprogrammed. If you know of any, please show me. <

You ask for animal behavior that could not have been preprogrammed. But I’m not sure any finite observed behavior could meet that standard.

For any finite sequence of behavior, it seems possible in principle to posit a sufficiently elaborate fixed program that produces it. The same would apply to human behavior.

So if the criterion for sentience is behavior that could not possibly have been preprogrammed, it is unclear how sentience could ever be established behaviorally, including in humans.

But the argument is that behavior that may just as well be preprogrammed and then executed mindlessly is not evidence of consciousness (because it is, again, mindless).

This seems to be circular reasoning:
Your reply seems to move from “the behavior could have been preprogrammed” to “the behavior is therefore mindless.” But those are different claims.

“Preprogrammed” describes how the behavior is generated. “Mindless” additionally says that no phenomenal experience accompanies that process. That does not follow unless we already assume that non-universal or preprogrammed computation cannot support phenomenal experience. Which is precisely the connection my criticism questioned.

So showing that animal behavior could be preprogrammed may show that the behavior does not prove sentience, but it does not by itself provide evidence of insentience.

In the reasoning you link in support of animal insentience, you argue that sentience and the capacity for universal creativity/critical thought come as a package deal. Dreaming seems to provide evidence against that connection.

Ordinary dreams can involve vivid visual experience, fear, pleasure, pain, etc., while metacognition, reality testing and critical reflection are greatly diminished. Lucid dreaming makes the distinction especially interesting: metacognitive awareness can return during an already conscious dream. The dream does not become phenomenally conscious only when lucidity begins; rather, an additional capacity for recognizing and critically reflecting on the experience appears.

This suggests that phenomenal experience can occur independently of the active critical capacities you take animals to lack.

You propose in your writings that universal creativity can emerge from lower-level, mindless computation, and that consciousness is among the emergent properties associated with creativity. But why must phenomenal experience emerge only together with universal creativity?

The fact that simple execution-only systems are not conscious does not show that no noncreative computation can have phenomenal properties. For example, pain could in principle have emerged from a different kind of lower-level computation long before that system acquired universal creativity.

So unless there is an explanation for why phenomenal experience cannot emerge from noncreative computation, the inference from “animals are noncreative” to “animals are insentient” does not seem to follow.

The inference would seem more natural in the opposite direction: evidence that animals lacked phenomenal experience might support doubts about more general mental capacities, but the absence of universal creativity does not by itself explain why even pain or color experience should be absent.