Activity Feed

should_adopt? and pending_criticisms are implemented in terms of each other. Doesn’t that cause an infinite regress?

Conjecture: a source of variation with its own knowledge can remain blind and external to its target while nonetheless receiving feedback from the target, so long as the feedback is non-specific to any content in the target.

Creativity (or “creation” or non-inferential novelty) is occurring in a system iff the system is being altered by a second system that is epistemically separated from it (its changes are nonspecific to the information content of the first system).

But if the source of variation only makes changes that are blind–that do not depend on the specific content of the target–then its implication space remains separate from the target’s. It is changing the target's implication space (its set of counterfactuals...), serving as a source of novelty for the target, while its own implication space is unchanged.

If changes are not blind, the changer is changing based on the content of the knowledge in the target, in which case the two have a joint implication space: their possible states are all implicit in the system, and there are no states that are novel with respect to either of them; creativity is not possible. It’s like two differently charged conductors coming into contact and neutralizing the electric field between them.

That these external sources of variation in the case of DNA do not themselves contain knowledge is happenstance, another special case. A source of variation acting on a given set of knowledge can be knowledge-bearing while still being epistemically external to the target (making only blind changes to it), so long as the variations do not depend on the specific content of the knowledge in the target. So: set A is altered by set B, each are knowledge-containing, but knowledge in B does not pertain to A.

For the changes made by a source of variation to be dependent on the content of any knowledge in the target, the source would have to contain knowledge itself. (The DNA mutation sources do not.) Blind changes are dependent on the medium in which information is contained, like an A versus a G in a strand of DNA (assuming these are information media). But the change made to the A or G by the cosmic ray is independent of the information they are hosting, if any. Any changes by the ray are dependent on the physical/chemical structure of the target, the substrate. Information is substrate independent. It is not the information stored in the A or G that is determining what changes are made to them.

The externally-derived sources of variation/novelty in the case of DNA (cosmic rays and failures in copying due to chemistry) are blind: they do not interact (epistemically) with the knowledge of the system.

Conversely, non-inferential novelty does not follow by any inference or deduction; it is not implicit in existing knowledge, so it is not in any set of knowledge's implication space (except that of the laws of physics...).

Knowledge which is novel in this way is what constitutes creation in the sense we mean in epistemology. The two known examples of non-inferential knowledge growth are biological evolution and some of the thinking which takes place in the minds of people (human or AGI).

Any given set of knowledge cannot grow itself non-inferentially. There must be a source of novelty, of variation, that is epistemically external to it.
If we take that claim as general, then the mutation of DNA constituting the creation/novelty/knowledge-growth in biology is a specific instance: cosmic ray strikes and errors in copying due to chemistry are sources of variation not caused by the DNA’s knowledge).

Claim: It is useful (if not crucial) to distinguish between two notions of novelty: inferential and non-inferential.

Knowledge that is inferentially novel may never have been seen before, but was already implicit in existing knowledge; it results from application and composition of existing knowledge (e.g. the running of programs). What is novel in this way can be instantiated without any external epistemic (knowledge-related) influence. A program needs infrastructure to allow its computation to occur; it does not need another program, in an epistemic sense: If there's a dependency, we should consider the two to be part of the same "epistemic system." Let's say the dependent target has no "implication space" until its needed source is causally linked to it. The source may have its own implication space. Once "joined," a new implication space is defined.

Inferentially novel output (the instantiation of knowledge in an implication space) is all that current computer software (and all AI) is capable of. These systems "create" knowledge only in this sense.

Assuming Sam did mean to criticize my opposition to assigning strengths to criticisms:

The issue isn’t that we don’t have a way to fix the hole. (We often do.)
The issue is that we have no rigorous way to tell how big the hole is. So, rather than evade criticisms by saying ‘but it’s only a small hole’, I’m saying we should always either fix holes in the boat, no matter their size, or choose a different boat.

Unclear what the boat stands for.

Sam criticized my idea, so presumably he means for the boat to represent #4711. But the way his criticism is phrased, it sounds more like the boat represents the methodology of assigning strengths to criticisms.

I wonder if there is any significance to the "distance" the laws of physics are from us, in the sense of emergence. Each quantum application of the laws of physics (conceived as discrete steps) is so far down, but maybe it need not be so. Maybe there are people in other laws of physics, where the laws are much closer at hand. Do the laws need to have a certain minimum range of emergence for personhood to be possible..? Couldn't people like us exist in a universe where fundamental physics is no more complex than the operation of our minds, or is the "breathing room" we have here necessary?

To be clear, the boat represents the idea that we should judge criticisms by their strength?

if somebody suggests some system they can’t put into practice, then the system is no good.

Agreed. My point is that you can't refute a flaw by saying the flaw doesn't have a workable system to resolve it. "Your boat has a hole in it". "Yeah, well I don't have any way of fixing it". That doesn't resolve the criticism of the boat. If it's unfixable and a real flaw, you might just need a new boat.

Logically, yes. But in practice, it won’t be the same.

For example:

Idea A: ‘Newtonian gravity is true.’ → Has pending criticisms (some predictions don’t match observation).
Idea B: ‘Newtonian gravity is not true.’ → Also has pending criticisms: ‘Not helpful’; ‘Doesn’t tell us how gravity works’; etc.

Related: https://www.daviddeutsch.org.uk/2014/08/simple-refutation-of-the-bayesian-philosophy-of-science/

Maybe I’m misunderstanding you, but if somebody suggests some system they can’t put into practice, then the system is no good.

This sounds like a version of #2195, see criticism #2198.

Then the idea ‘I could either address that pending criticism or, if I fail to do that, look into all the other pending criticisms’ would have no pending criticisms.

If other criteria for the selection of conjectures were built into the system, such as in the genes of a human, the growth of knowledge would be bounded to only conjectures meeting those criteria, even if the variation/conjecture aspect of the process were proceeding via a random process with no built-in knowledge. Fixed criteria means bounded knowledge, means no universality. So for unbounded progress (universality), the criteria must also be varied. But how are they criticized? What criteria for their selection can there be to avoid a regress, except survival?

All subsequent knowledge in the system will be descended from the initial knowledge. So all knowledge in a knowledge-growing system must be either created by or comprised of replicators.

Seeking criticism of various ideas relating to ongoing research into AGI, creativity, personhood and open-ended evolution.

Knowledge growth (such as in genes in the biosphere, or in a person) can only proceed by conjecture and criticism -- variation and selection. In a system which can create knowledge, but has not yet, there are no modes of substantive criticism available, since that would itself require knowledge. So how can the system create knowledge at all? Because although substantive criticism is not yet possible, the crudest form of full-on refutation is: the destruction of the information medium on which a conjecture is based, i.e., death. Death is the only "undesigned" refutation. So any initial knowledge created in an otherwise empty knowledge-creating system can only be that relating to how to avoid death in the local environment. This is the knowledge which defines replicators.