Can qualia be separated from personhood?
Can a program which is not a person constitute an experience?
Imagine you are in a pitch black room. Before your eyes, a spotlight illuminates an apple for 5 seconds before darkness returns. Among other things, your mind will render the image of the apple for the 5 seconds, then it will not (afterimages aside). Assume the physical stimulus is identical for the whole 5 seconds.
Itemized discussion below.
Log in or sign up to participate in this discussion.
With an account, you can revise, criticize, and comment on ideas, and submit new ideas.
(3) The programs rendering the apple imagery must be looping until stopped, since they could not have advance knowledge of when the stimulus stops.
(4) The programs rendering the apple are not people, so cannot themselves constitute experience.
(5) Repeated running of the same fixed program, not being a person, does not make it a person.
(6) Repeated running of the same fixed program is automatic, requires no creativity, and cannot constitute experience.
PROBLEM: Why are we conscious of the apple rendering? Given (6), why is there an experience of it, if the programs comprising it are looping, and so are therefore predefined?
SOLUTION: The apple programs are not the same programs one execution to the next. They are being re-evolved every time they are run. This evolution is what the person is doing, and so must be what gives rise to the experience consisting of the apple rendering.
This suggests that all experience is determined by what programs are being subjected to evolution at any given time, the niches that are being adapted to. But why is not all creativity in the mind conscious? (All consciousness might necessarily be creativity).
Either there is no unconscious creativity, or only evolutionary/creative epochs with certain properties are conscious. The most obvious candidate for the property is complexity (in the sense of sophistication): only programs (existing knowledge) of a certain sophistication, once subjected to the evolutionary process, necessitates consciousness. Complex problem solving seems to require consciousness. Meanwhile, we do not seem to be conscious of "simpler" creative tasks, like... Like what? What is a "simple" creative task? What is an example of a creative task we perform unconsciously? How could we determine it was an act of creation (new knowledge), and not an act of deductive inference of the kind characterizing AI?
"No unconscious creativity" seems the simpler option. But here we arrive again at biological evolution, which is unconscious, yet is creating knowledge. Does this serve as a distinction between explanatory knowledge and non? Explanatory knowledge can only be created by a conscious process?
A person could create the same knowledge that biological evolution does, if only by simulating it. But it could still be true that only people can create explanatory knowledge. (That they can create all possible explanatory knowledge is Deutsch's criterion for personhood.)
only people can create explanatory knowledge
How is an LLM not creating new explanatory knowledge (even if worse than the existing, by any measure), by varying some existing written explanation? It could even vary and select by some criterion of its "choice", thus realizing Popperian epistemology.
Whatever new "explanations" it creates are derivable from (and by?) the knowledge in the training data. It isn't evolution if all of the variations and selection criteria are stored ahead of time. That's just a search process, as in the case of Move 37 per AlphaGo.
If only some of the criteria are stored, and the rest are random, is it still evolution? Is evolution only happening if there is random variation? But we could program an LLM to do that as well...
But an AI programmed to make random variations to its conjectures (English or otherwise) can only do so by choosing from an existing set of variations. Again, that knowledge is pre-existing. True evolution must involve variations to the substrate on which the knowledge is based; variations must be agnostic to the semantics of whatever they are acting upon, else they are already implicit from it, in which case their application does not constitute a truly novel conjecture (in the sense defining creativity).
Even if variations are agnostic to any meaning or context of the knowledge, why are they still not implicit? Anything is implicit from anything else, if implicit just means: follows from when a given change is applied... The whole question is where the change is coming from... (?)
Yes, everything is not implied by everything else, so I think what we must mean by implicit is: can be deduced from/assembled using available transformations.
For knowledge to be truly novel in the sense of having come from creativity, it must not be deducible. Ambient, unjustified substrate is "taken from the environment" and filtered by selection. What survives can be increasingly truth-containing.
Mutations to a substrate, meaning blind mutations, not specific or designed, must not be implicit from the substrate; the result of their application cannot be deduced in any way... Otherwise the knowledge they might contain would already have been present...
#4806 is saying: variations of knowledge being agnostic to that knowledge's meaning means they are not implicit from it, else implicit doesn't mean anything. So #4806 is only really asking if what matters is the source of knowledge, and that isn't really a criticism of #4805.
Criticism #4875 applies to #4806, as shown.
To clarify and add on to #4805: No, we couldn't program an LLM (on its own) to do random variation in the sense constituting evolution, because all of the randomly chosen changes to its outputs are still implicit from its current knowledge (training data + design from programmers). There is also no means of criticism that are not also implicit: any niche or criterion it generates, then seeks to satisfy, was derived again from its existing knowledge. It is a closed system (whether or not we have run it such as to reveal everything it implies!).
This implies that no two instances of experience, even if seemingly identical, are caused by the same programs.
Actually this is not implied. One experience and an identical later one could be caused by the same program(s) being run again at a later time; if the program which is identical to the given experience is part of an "evolutionary personhood program", that still qualifies: If the second experience is identical, under the above solution that just means that the exact same evolutionary steps are taken in the second case. Maybe this would virtually never happen, but poses no problem of principle.
But why would the system ever re-evolve to the satisfaction of a niche already satisfied previously? If the programs evolved by the evolutionary aspect of the person already exist, there is no more need for evolution of them.
Because programs present in the system at one time could be no longer present at another time. Previously well-adapted programs could have decayed, been destroyed or consumed. So the same evolutionary path (approximately or not) could be travelled again, in principle.
The system may not have perfect knowledge of all programs present in it. The repeated independent emergence of winged flight in the biosphere comes to mind.
Is all conscious experience not the running of programs, but computation that is realizing the evolution of programs? Computation which cannot be abstracted to any program, then? So in what sense can a person "be programmed"? Is personhood computational, but "non-programmatic"?
By the Church-Turing Thesis, all computation can be specified/programmed. So the evolutionary aspect of a person can be specified/programmed, if it is computational.
Programs could be evolved non-computationally. But that process could itself still be simulated, per the Church-Turing-Deutsch Thesis.
It could be simulated, but maybe it's very hard/intractable to do so. Maybe personhood harnesses physics to do the evolving, like a windmill harnesses the wind. Programs implemented such that the laws of physics cause them to evolve (unboundedly)?
But if the evolution is the defining feature of personhood, and the evolution is non-computational, then the personhood is non-computational. And consciousness would then not be a software property.
SOLUTION: The apple programs give rise to consciousness only in a given context. Only when run a certain way (by a person).
This suggests that programs can be “run differently” to result in a different computation. This is false because it violates Substrate Independence: the instantiation of a program is unaffected by its physical implementation. If a “context” changes what the program is computing, then that’s a different program. Suggesting that a person running the apple programs “makes them” conscious therefore is not sound. The programs are either conscious or not. If they were, by (A1), they would be people.
Assumption A1: Only programs that are people can, while running, constitute qualia/experience/subjectivity/consciousness.
A quale can be recorded and replayed later, arbitrarily many times. That looping program is not a person: it can only create whatever knowledge it originally did, not any possible knowledge, maybe until freed from the loop. Yet it still constitutes experience.
So people are not the only programs that can be qualia.
A quale can be recorded and replayed later, arbitrarily many times.
How do you know this?
Assuming qualia are classical, there is nothing stopping every step in the computation comprising them from being measured, like any computation. Then, nothing is stopping that recording/program from being rerun.
This is true, but assumes that "qualiation" still occurs when each subsequent state is pre-existing and determined. Do we know this? What if qualiation somehow depends on the (computational) indeterminacy of the subsequent states..?
Is a quale a sequence of information states? Is this true of any given computation? Does the quale "care" if the states are computed afresh or read from memory?
That's what I get for suddenly returning to this discussion after a long gap. I was thinking that any computation can be recorded and re-run at will, in principle. So qualia can too. But indeed, I have suggested in this discussion that it is the evolution of programs that causes experience, and evolution cannot be scripted. This means a quale that is recorded will not "qualiate" if it is rerun exactly as recorded.
TL;DR: I don't.
My suspicion remains that creativity and qualia come from some kind of blind exchange between layers of emergence, of which mutations and mistakes in copying, as seen in DNA, are an example.
Actually maybe this isn't so obvious. A person need not create new knowledge for the personhood property to be present, since it is a counterfactual property: a person can create new knowledge (new and non-inferable, I argue).
True that a person need not create new knowledge, but a looping quale is not a person, per #5035, so the claim survives this criticism.
If qualia are only present when new knowledge is being made, then for a quale to be replayed it could be that there must be a reset each time: the knowledge produced from the previous run is erased. So w.r.t. the AGI system alone, new knowledge is being created each time (it doesn't matter if the knowledge is copied from a previous run and stored elsewhere, in which case it is not new w.r.t. that combined system).
A looping quale is also not a person because personhood is not just: can create a given piece of new knowledge, it is: can create any knowledge (not just that created by this quale; again assuming knowledge creation defines qualia).
Should clarify that personhood is: can create any new knowledge in principle (excluding resource constraints). A program consisting of a looping quale cannot create any possible knowledge, whether or not it is creating some each time it's run.