Tyler Mills

My suspicion remains that creativity and qualia come from some kind of blind exchange between layers of emergence, of which mutations and mistakes in copying, as seen in DNA, are an example.

That's what I get for suddenly returning to this discussion after a long gap. I was thinking that any computation can be recorded and re-run at will, in principle. So qualia can too. But indeed, I have suggested in this discussion that it is the evolution of programs that causes experience, and evolution cannot be scripted. This means a quale that is recorded will not "qualiate" if it is rerun exactly as recorded.

TL;DR: I don't.

True that a person need not create new knowledge, but a looping quale is not a person, per #5035, so the claim survives this criticism.

True, knowledge could be created from each run of a quale, but a looping quale isn't a person, per #5035, so the claim in #5031 that not only people can undergo experience still stands.

Should clarify that personhood is: can create any new knowledge in principle (excluding resource constraints). A program consisting of a looping quale cannot create any possible knowledge, whether or not it is creating some each time it's run.

A looping quale is also not a person because personhood is not just: can create a given piece of new knowledge, it is: can create any knowledge (not just that created by this quale; again assuming knowledge creation defines qualia).

If qualia are only present when new knowledge is being made, then for a quale to be replayed it could be that there must be a reset each time: the knowledge produced from the previous run is erased. So w.r.t. the AGI system alone, new knowledge is being created each time (it doesn't matter if the knowledge is copied from a previous run and stored elsewhere, in which case it is not new w.r.t. that combined system).

Actually maybe this isn't so obvious. A person need not create new knowledge for the personhood property to be present, since it is a counterfactual property: a person can create new knowledge (new and non-inferable, I argue).

A quale can be recorded and replayed later, arbitrarily many times. That looping program is not a person (no knowledge is created, however many times it loops, for instance), but it still constitutes experience.
So people are not the only programs that can be qualia.

Are pacifism, socialism and compromise all short-termist?

Pacifism reduces casualties --- in the short term.

Socialism reduces poverty --- in the short term.

Compromise reduces disappointment --- in the short term.

Not if personhood requires literal explanatory universality, in the sense of: can create any explanation, since there are explanations which can only be created by a Turing complete system..., surely.

I've been thinking about the principle that principles should not or do not admit of exceptions, else they are not principles. Yet, I've thought, we seem to not abide by this, even with something as fundamental as freedom of speech. We make exceptions for calls to violence, doxing, etc.

But I've (somehow only) just realized that those are not what we might call "standalone exceptions" -- they are overlaps with other principles, those of non-initiation of force and privacy, in this case.

So we can say: principles do not admit of exceptions, but if there is more than one principle in the system, one's exception can be another's enforcement. Which is not an inconsistency. Then the question is, which one is the higher priority? Maybe this is where the court system comes in. (I don't know much about politics).

This is the only solution to the "apple problem" raised in #4752.
(Criticize this with alternatives solutions).

To clarify and add on to #4805: No, we couldn't program an LLM (on its own) to do random variation in the sense constituting evolution, because all of the randomly chosen changes to its outputs are still implicit from its current knowledge (training data + design from programmers). There is also no means of criticism that are not also implicit: any niche or criterion it generates, then seeks to satisfy, was derived again from its existing knowledge. It is a closed system (whether or not we have run it such as to reveal everything it implies!).

#4806 is saying: variations of knowledge being agnostic to that knowledge's meaning means they are not implicit from it, else implicit doesn't mean anything. So #4806 is only really asking if what matters is the source of knowledge, and that isn't really a criticism of #4805.
Criticism #4875 applies to #4806, as shown.

Yes, everything is not implied by everything else, so I think what we must mean by implicit is: can be deduced from/assembled using available transformations.

For knowledge to be truly novel in the sense of having come from creativity, it must not be deducible. Ambient, unjustified substrate is "taken from the environment" and filtered by selection. What survives can be increasingly truth-containing.

Mutations to a substrate, meaning blind mutations, not specific or designed, must not be implicit from the substrate; the result of their application cannot be deduced in any way... Otherwise the knowledge they might contain would already have been present...

Upon review, we should maybe say instead that personhood should not be defined solely in terms of tractability, which the bounty terms are not clear about. As it stands (bounty aside), I find myself still seeing tractability as an important aspect of epistemology and the mystery of personhood/knowledge creation, a hunch reinforced as I continue reading through "Why Philosophers Should Care About Computational Complexity" by Scott Aaronson: https://www.scottaaronson.com/papers/philos.pdf