Misc. Conjectures Re: AGI and Evolution
Seeking criticism of various ideas relating to ongoing research into AGI, creativity, personhood and open-ended evolution.
Log in or sign up to participate in this discussion.
With an account, you can revise, criticize, and comment on ideas, and submit new ideas.
Knowledge growth (such as in genes in the biosphere, or in a person) can only proceed by conjecture and criticism -- variation and selection. In a system which can create knowledge, but has not yet, there are no modes of substantive criticism available, since that would itself require knowledge. So how can the system create knowledge at all? Because although substantive criticism is not yet possible, the crudest form of full-on refutation is: the destruction of the information medium on which a conjecture is based, i.e., death. Death is the only "undesigned" refutation. So any initial knowledge created in an otherwise empty knowledge-creating system can only be that relating to how to avoid death in the local environment. This is the knowledge which defines replicators.
Natural selection requires differential replication, not death. (Death is just an extreme form of differential replication, when compared to any allele that does replicate.)
Maybe natural selection logically does not require death, but when we introduce a scarcity of resources, isn't death an inevitable byproduct? Eventually, the best replicating will be those which take the resources from others? If no death, then number of entities increases without bound. So any real system will have it, or will stop functioning when the limit is reached.
It's sort of like: the "this is replicating less" criticism has no actual weight to it unless there is a point beneath which any less fecundity = bye-bye.
**If things with arbitrarily low fecundity are left intact forever, how is differential replication a criticism?**
The criticism "this is replicating less" should be generalized to "this allele/information has less causal power than an alternative." That has tremendous weight.
There is a natural preference for the one with more causal power, it is naturally selected over alternatives.
However, it is not clear to me how you are "weighing" these criticisms. So maybe my weighing isn't relevant to your weighing.
By "weight" I meant that there being fewer instances of an idea (or less causal power, as you propose), doesn't actually criticize anything (content-wise), until the idea drops to zero (at which point, that idea is gone, and the knowledge content in the system has meaningfully changed -- epistemically changed...). Extinction (death of all instances -- and alleles..?) does seem like a refutation of the idea/gene.
This is the same as in an economy, seems to me. Maybe part of what can be clarified in this discussion will be these different modes of criticism. Please see #5133 and related.
Scarcity of resources should not be assumed. You want underlying necessity of the mechanism, not a strained analogy. There is no scarcity for idea resources.
But couldn't scarcity of resources be part of the underlying mechanism? We wouldn't want to say, "assume no speed limit to mass, because we want the underlying necessity of the mechanism of gravity." But the scarcity of speed is integral to the mechanism.
no scarcity for idea resources
"Idea resources" are computational resources, which are always scarce. Even with universality (modulo resources), the limits of the resources must have some effects on the evolution, I would think. Some developmental paths cannot be taken.
In addition to scarcity, you have to assume unbounded growth of the population in an ecosystem, which does not happen.
Does not happen? With unlimited resources, why can't it? Competition will not always prevent it, will it?
The term "death" is being used too vaguely. (Or else I am misunderstanding.) Do you think that a gene replicating and then dying (while its offspring survives) is "death?" A "refutation?"
Or do you mean "total elimination of the allele" as "death" or "refutation?"
I've been thinking of death as the loss of an instance of an idea from the soup. Part of what I'm curious about here is the epistemic difference between deaths of instances, and deaths of the whole lineage (which seems like a "refutation" in a different, more meaningful sense). See #5167.
I think it is a mistake to say or imply that variation and selection IS conjecture and criticism. At best C&C is the subset of V&S that occurs in minds. The biosphere doesn’t actually conjecture anything.
Why aren't mutations made to DNA a form of conjecture? I think of conjecture as being a pretty general term -- isn't all knowledge conjectural, ours or otherwise? Maybe semantics. Agreed that criticism is a subset of selection: selection that is a function of the knowledge in the target, perhaps? Whereas differential replication is how selection proceeds without referring to the specific knowledge in the target..? [Overall idea stands despite my concession above]
All subsequent knowledge in the system will be descended from the initial knowledge. So all knowledge in a knowledge-growing system must be either created by or comprised of replicators.
If other criteria for the selection of conjectures were built into the system, such as in the genes of a human, the growth of knowledge would be bounded to only conjectures meeting those criteria, even if the variation/conjecture aspect of the process were proceeding via a random process with no built-in knowledge. Fixed criteria means bounded knowledge, means no universality. So for unbounded progress (universality), the criteria must also be varied. But how are they criticized? What criteria for their selection can there be to avoid a regress, except survival?
Claim: It is useful (if not crucial) to distinguish between two notions of novelty: inferential and non-inferential.
Knowledge that is inferentially novel may never have been seen before, but was already implicit in existing knowledge; it results from application and composition of existing knowledge (e.g. the running of programs). What is novel in this way can be instantiated without any external epistemic (knowledge-related) influence. A program needs infrastructure to allow its computation to occur; it does not need another program, in an epistemic sense: If there's a dependency, we should consider the two to be part of the same "epistemic system." Let's say the dependent target has no "implication space" until its needed source is causally linked to it. The source may have its own implication space. Once "joined," a new implication space is defined.
Inferentially novel output (the instantiation of knowledge in an implication space) is all that current computer software (and all AI) is capable of. These systems "create" knowledge only in this sense.
How much overlap is there between the concept of inferential novelty and reach, if any?
Fun question. Say the reach of a program is the set of all problems it can solve or help solve (whether anyone knows, or whether the problems exist yet). All of that program's possible outputs together comprise the implication space -- it sounds like "range" is the actual term. The instantiation of each item in the range is an occurrence of my "inferential novelty". The range is implicit in a program when it is created, but not all of that output will help solve problems. The program's reach is the subset that does.
Maybe this definition accommodates both the multiversal and atemporal aspects of reality: a program's reach is the set of its outputs that solve problems in any universe (past, present or adjacent).
But this depends on what a "problem" is; any output will be the solution to some problem, somewhere, so all programs would have unlimited reach.
So reach should only ever be invoked with respect to existing knowledge? So a program's reach is the intersection of its codomain and the solutions to known problems.
The appropriate term is range, not codomain. Correcting that and the redundancy per #5189: A program's reach is the intersection of its range and the solutions to current problems.
I think all problems are known, so we can just say "solutions to problems."
A problem is a conflict between theories, meaning it relates to knowledge currently instantiated.
Conversely, non-inferential novelty does not follow by any inference or deduction; it is not implicit in existing knowledge, so it is not in any set of knowledge's implication space (except that of the laws of physics...).
Knowledge which is novel in this way is what constitutes creation in the sense we mean in epistemology. The two known examples of non-inferential knowledge growth are biological evolution and some of the thinking which takes place in the minds of people (human or AGI).
Any given set of knowledge cannot grow itself non-inferentially. There must be a source of novelty, of variation, that is epistemically external to it.
If we take that claim as general, then the mutation of DNA constituting the creation/novelty/knowledge-growth in biology is a specific instance: cosmic ray strikes and errors in copying due to chemistry are sources of variation not caused by the DNA’s knowledge).
Did you mean "Any given set of knowledge cannot grow itself inferentially"?
No, my thinking here is that any given set of knowledge can only do inference: apply what transformations it has to whatever data it has, and that's it. It can't output things that are not already implicit in it, so cannot grow, in any epistemic sense. Even if running some kind of evolutionary procedure. An external source of novelty is required for growth, with respect to any given set of knowledge.
The externally-derived sources of variation/novelty in the case of DNA (cosmic rays and failures in copying due to chemistry) are blind: they do not interact (epistemically) with the knowledge of the system. That is, their changes are independent of the knowledge in the system. This is what makes these sources epistemically separate.
For the changes made by a source of variation to be dependent on the content of any knowledge in the target, the source would have to contain knowledge itself. (The DNA mutation sources do not.) Blind changes are dependent on the medium in which information is contained, like an A versus a G in a strand of DNA (assuming these are information media). But the change made to the A or G by the cosmic ray is independent of the information they are hosting, if any. Any changes by the ray are dependent on the physical/chemical structure of the target, the substrate. Information is substrate independent. It is not the information stored in the A or G that is determining what changes are made to them.
That these external sources of variation in the case of DNA do not themselves contain knowledge is happenstance, another special case. A source of variation acting on a given set of knowledge can be knowledge-bearing while still being epistemically external to the target (making only blind changes to it), so long as the variations do not depend on the specific content of the knowledge in the target. So: set A is altered by set B, each are knowledge-containing, but knowledge in B does not pertain to A.
If changes are not blind, the changer is changing based on the content of the knowledge in the target, in which case the two have a joint implication space: their possible states are all implicit in the system, and there are no states that are novel with respect to either of them; creativity is not possible. It’s like two differently charged conductors coming into contact and neutralizing the electric field between them. An "epistemic capacitor."
But if the source of variation only makes changes that are blind–that do not depend on the specific content of the target–then its implication space remains separate from the target’s. It is changing the target's implication space (its set of counterfactuals...), serving as a source of novelty for the target, while its own implication space is unchanged.
Creativity (or “creation” or non-inferential novelty) is occurring in a system iff the system is being altered by a second system that is epistemically separated from it (its changes are nonspecific to the information content of the first system).
Conjecture: a source of variation with its own knowledge can remain blind and external to its target while nonetheless receiving feedback from the target, so long as the feedback is non-specific to any content in the target.
An example of this is prices in a market: a price communicates information about (or caused by) knowledge in a system while being agnostic to the content; the specifics and interactions of the target knowledge are abstracted away, so any changes made by sources receiving the price information remain blind to the target, and therefore can remain a source of novelty to them (the prices themselves are not a source of novelty; they are a means of retaining a causal relationship while preventing the “collapse” of two epistemic systems into one)...
Any relation in all this to entanglement and decoherence? In the picture laid out here, the non-specificity of feedback seems to be preventing decoherence of a sort...
I.e., information caused (if only partially) by some source can be "content independent" w.r.t. that source in a similar sense to how information itself is substrate independent.
Cosmic rays have been known to flip bits in computers, eg impacting election results: https://ui.adsabs.harvard.edu/abs/2021arXiv210505103N/abstract
However, due to computational universality, we should be able to simulate a knowledge-growing system plus cosmic rays impacting that system.
So then at that point the source of the mutation is part of the program again…
Specific criticisms, those taking into account the content of their target, are always discrete and absolute because they identify a contradiction. (E.g. criticisms of a scientific theory.)
But the extinction of a replicator by destroying its last instance does seem absolute, unlike the change in its population, which I suggest is non-specific to the content in #5133.
Non-specific criticisms, those not taking into account the content of their target, are always "analogue" (or quantitative as opposed to qualitative) in the sense of being based on a variable whose quantity changes in a medium distinct from that hosting the knowledge it is serving as criticism for. (E.g. capital allocation in an economy; number of instances of a replicator.)
The capital allocated to a business is non-specific to its content? Conflating with the prices of a good?
The capital a business receives from a buyer is feedback that is non-specific to the content of the product or the business which produced it. It's not knowledge bearing. It's a fungible, universal form of feedback, like energy -- and like the number of instances of a replicator..?
The change in population of a replicator is non-specific to its content?
How is the quantity of a replicator not based in the same medium as the replicator?
None of the replicators have that information. If that quantity variable exists, it's instantiated somewhere else.
Wouldn't it have to be instantiated in the overall system itself, which contains the replicators (among other things?)? Otherwise, how could that variable be a form of criticism by the system?
Yes, the quantity variable is instantiated in the collective physical substrate comprising all of the replicator instances.
Would it help to distinguish between dogmatic vs. critical information processing?
That’s a distinction I’ve been using. All programs written to date are dogmatic. Whereas a genuine evolutionary algorithm would be critical, at least in part.
We don’t know how to write critical code. We only know how to write dogmatic code that looks critical.
This framing seems good, sure. So what does "critical" really mean?
Does it mean: "Can change an existing thing"..? But that criticism program is dogmatic w.r.t. itself, isn't it? This is the whole question, here (or I'm confused), in your terms:
Can criticism, the variation of a dogmatic program, only happen at the hands of another dogmatic program? There's only "externally facing" criticism..? There are no critical programs? Only dogmatic ones criticizing other dogmatic ones.
But then there would need to be a preprogrammed dogmatic program at the bottom. (Or would there..?)