Hard to Vary or Hardly Usable?
Showing only #3069 and its comments.
See full discussionLog in or sign up to participate in this discussion.
With an account, you can revise, criticize, and comment on ideas.My critique of David Deutsch’s The Beginning of Infinity as a programmer. In short, his ‘hard to vary’ criterion at the core of his epistemology is fatally underspecified and impossible to apply.
Deutsch says that one should adopt explanations based on how hard they are to change without impacting their ability to explain what they claim to explain. The hardest-to-change explanation is the best and should be adopted. But he doesn’t say how to figure out which is hardest to change.
A decision-making method is a computational task. He says you haven’t understood a computational task if you can’t program it. He can’t program the steps for finding out how ‘hard to vary’ an explanation is, if only because those steps are underspecified. There are too many open questions.
So by his own yardstick, he hasn’t understood his epistemology.
You will find that and many more criticisms here: https://blog.dennishackethal.com/posts/hard-to-vary-or-hardly-usable
I think the first question is whether HTV is a real concept (because if real, it is programmable, and via EC to arbitrary precision)
To understand if it’s real, we need to seek counterexamples/ counterarguments, not demand that a program can be written
What would such a program prove ? Not that HTV is real, but also not that we understand something about HTV.
That’s because Deutsch only says : no program = no understanding. That implies having a basic conception programmed can mean that you understand something. Take the season’s example, you could simulate that replacing Gods would not change the fact that they cry but that tears are not the same as rain etc. Granted, this would only be for 1 example, extending HTV to general examples would be needed. But with such basic program, for 1 example theory, we can’t conclude either that we do not understand anything about HTV.
Criticising HTV would anyway be the more important first step. Maybe examples of good theories with some ETV aspects (compared to rejected theories) in them could reveal some more.
Criticising HTV would anyway be the more important first step.
The linked blog post has several criticisms of HTV.
Do you mean "HTV is underspecified by Deutsch" ? But that is not a criticism ? It does not point to a mistake/ contradiction with HTV ?
It’s a criticism. Deutsch says to use HTV but never explains in sufficient detail how to do that.
How is that a criticism ? What mistake does it point out/ argue for ?
The mistake is insufficiency. If someone gives you a recipe for baking a cake but doesn’t specify ingredients or bake time, that’s a problem.
Isn't every theory infinitely underspecified ? Also, I would think that criteria for sufficiency must always be subjective ones (e.g. a working computerprogram cannot be itself a proof of meeting an some objective sufficiency criterium)? So I don't see how insufficiency points to a conflict of ideas/ contradiction
Also, I would think that criteria for sufficiency must always be subjective ones (e.g. a working computerprogram [sic] cannot be itself a proof of meeting an some objective sufficiency criterium)?
No, there are objective criteria.
We’re getting off topic. I’m currently running a bounty requesting a working implementation of HTV.
If you think you can beat the bounty, do it. I’m not interested in anything else for now.
Isn't every theory infinitely underspecified ?
No. For example, the theory of addition is sufficiently specified: we have enough info to implement an algorithm of addition on a computer, then run it, test it, correct errors with it, and so on.
Yes. When you have program you can test a concept (incl. whether it is sufficiently defined to allow a program in the first place). But the other way around does not work: "If one does not have a program, then the concept is underspecified".
One way to program HTV could be to feed 2 explanations of the same phenomenon (in the form of text strings) to an LLM that is trained on seeking ETV patterns in text (things of the form "and then -all of a sudden- X happened ..." or "and Y (e.g. tears of a God) is kind of like Z (e.g. rain)" ) and seeking HTV patterns in text (e.g. Y happened because of X, with the LLM evaluating whether it is actual causation, whether if X did not happen, Y could not happen).And then the LLM could rank score the HTV-ness of each string (as a first approximation)
When you have program [sic] you can test a concept (incl. whether it is sufficiently defined to allow a program in the first place). But the other way around does not work: "If one does not have a program, then the concept is underspecified".
That isn’t what I said anyway. No disrespect but frankly I don’t think you know what you’re talking about.
I didn’t read the rest of your comment because you keep talking instead of coding. I’ll delete any further comments of yours that don’t contain code that at least tries to meet the bounty terms.
Isn't every theory infinitely underspecified ?
This stance is presumably a version of the epistemological cynicism I identify here.
I don’t think so, for two reasons. 1) Skepticism came long before Popper’s fallibilism.
2) Skepticism is too different from fallibilism to consider it a continuation.
"HTV is underspecified by Deutsch"
That isn’t a quote. Don’t put things in quotation marks unless they are literal quotations or obviously scare quotes.
Criticising HTV would anyway be the more important first step. Maybe examples of good theories with some ETV aspects (compared to rejected theories) in them could reveal some more.
That could work, yeah. What other criticisms of HTV can you think of?
I think your challenge asks for the wrong kind of thing. Deutsch’s “hard to vary” is a guideline for criticizing explanations, not a step by step decision algorithm. In this paper he says scientific methodology does not prescribe exact procedures, and that “better” explanations are not always totally rankable in a clean, mechanical way. “Hard to vary” mainly means avoiding explanations that can be tweaked to fit anything, because then they explain nothing, so the lack of a universal scoring program does not refute the idea.
THE LOGIC OF EXPERIMENTAL TESTS, PARTICULARLY OF EVERETTIAN QUANTUM THEORY
https://www.constructortheory.org/portfolio/logic-experimental-tests/
From the paper (p. 3):
An explanation is better the more it is constrained by the explicanda and by other good explanations,5 but we shall not need precise criteria here; we shall only need the following: that an explanation is bad (or worse than a rival or variant explanation) to the extent that…
(i) it seems not to account for its explicanda; or
(ii) it seems to conflict with explanations that are otherwise good; or
(iii) it could easily be adapted to account for anything (so it explains nothing).
Hey Fitz, welcome to Veritula.
I realize that DD doesn’t think of it in strict, procedural terms, but I just don’t think that’s good enough, for several reasons. One is that it’s too vague, as I explain here. We don’t know how to actually do anything he says to do, beyond broad suggestions.
Deutsch’s “hard to vary” is a guideline for criticizing explanations, not a step by step decision algorithm.
But he says to use hard to vary as part of a decision-making algorithm. As quoted in my blog post:
“we should choose between [explanations] according to how good they are…: how hard to vary.”
Brett Hall responds to the contradiction between HTV and ‘if you can’t program it, you haven’t understood it’: https://x.com/ToKTeacher/status/2090780593739243793
He agrees we can’t program HTV but says even if we don’t know how or why an explanation is HTV, we still know that it’s HTV.
The blog post already sidesteps that issue by granting the user the ability to input some HTV score without having to give any reasoning. That still leads to all sorts of trouble.
Dirk replies that the bounty doesn’t ask for essentialist definitions. It instead asks for clarity around how to actually use HTV:
Rejecting essentialism doesn't grant your ideas immunity from clarification. Being vague is not something to be proud of.
By your standard, I could invent any bullshit property of explanations, call it "glorpiness" and when you ask what it means, reply that definitions miss the point.
Brett replies, among other things: “Calling it ‘bullshit’ doesn’t win the argument. People have been doing that kind of thing with Popper, falsification and conjectural knowledge since the 1930s.”
Dirk replies that he “called "glorpiness" bullshit, not HTV” nor Popper. Brett completely missed the point of Dirk’s argument.
Brett’s response basically ignores existing criticisms and sounds like he’s lecturing people who haven’t read Deutsch or Popper. He’s forgotten that I’ve already corrected him on several misattributions of his to Popper and his misinterpretation of BoI re preference formation.
Brett says “Denying that good explanations of the physical world are hard to vary leaves us with *no better explanation* of what has driven progress in science, history and reason broadly.”
By saying HTV is central to progress, Brett contradicts his previous claim (#5355) that HTV is just a feature of explanations.
I reply that the actual explanation is that people who make progress participate in a tradition of criticism. They’re critical of ideas and try to address all (or almost all) known criticisms. That process may lead to ideas having the feature Brett calls hard to vary, but it’s strictly an effect of that critical activity, not its means.
I had previously linked Brett to my blog post containing that explanation. He liked that reply at the time so he’s definitely seen it, but he’s still ignoring it.
Brett replies saying “‘Hard To Vary’ is not vague. It is a real and useful solution to a philosophical problem. All actual scientific theories are hard to vary.”
He says it isn’t vague but then never gets specific about how to tell how hard to vary an explanation is or how to compare explanations this way. Instead, he mistakenly claims that isn’t the purpose of BoI, contrary to the evidence presented to him.
Brett says HTV stands in opposition to mysticism.
I think that was the intention behind it, but its effect has been the opposite, to an extent. Hand-wavy referrals to explanations being HTV and rationality being synonymous with creativity, without explaining in detail how these things work, turn them into a black box that may as well work magically to give us the answers we want.
Being rational requires a criterion of rationality that is clear and precise enough so you know when you stray from it. HTV isn’t.
Dirk replies to a bunch more points Brett has made. I won’t get into them, at least not for now. Brett’s previous arguments have too many pending criticisms, and he ignores them, so tracking additional points won’t help. Brett isn’t in truth-seeking mode, he’s in defend-Deutsch mode. If I see Brett addressing any pending criticisms on X after all, I’ll update V accordingly.
Brett says “Rational decision making is not a matter of pulling a lever and cranking through a calculation.”
Neil Hudson says in response to the bounty:
I wonder if this objection risks making a Hilbertian mistake: assuming that an objective criterion must provide a universal decision procedure. Gödel taught us not to equate objectivity with decidability. Why should HTV need to algorithmically rank every pair of explanations to be meaningful?
Not a mathematician but AFAIK, Hilbert basically asked if we could make a machine that answers every math question correctly. The answer to that is ‘no’. But an algorithm for rational decision-making isn’t about that.
Neil replies fallibility means the criterion must be able to get answers wrong or not deliver an answer at all sometimes. And if that’s the case, not any pair should be needed. Otherwise, “reliably compare” would be a demand for infallibility.
I reply:
The word “reliably” refers to the claim in BoI that ppl search for HTV explanations “systematically”; that they follow some yet-to-be-specified system to form preferences this way.
The system/program, being man-made, would of course be fallible. Still, since BoI says HTV is at the heart of all rational endeavors, the system must be universal. Hence the requirement for any pair.
More on the difference between a criterion of rationality and the demand for infallibility: #5336
Why should HTV need to algorithmically rank every pair of explanations to be meaningful?
Because, per BoI, we can’t claim to have understood a computational task if we can’t program it. The formation of rational preferences is a computational task. I say this in my reply.
I also reply that it needs to rank any pair, not every pair.
Several people have replied to the bounty saying it’s asking for the wrong thing; that HTV is only about rejecting ETV explanations, and that it’s only one mode of criticism among many.
They’re confused about what BoI actually says about HTV. It says HTV is central to the enlightenment and all rationality, that people use it systematically, that we should choose between explanations based on how hard to vary they are, and so on. According to BoI, HTV is not just another mode of criticism. It’s the central mode explaining progress.
As I say here, I’m personally on board with adopting a weaker form of HTV (or really ETV) where we only treat it as one mode of criticism among many and reject explanations if they have arbitrary components. But again, that’s not what BoI says. The bounty is about taking BoI seriously and rehabilitating HTV as presented there.
Neil Hudson says the bounty “would require programming the creative process by which relevant variations are conjectured in the first place.”
I reply that “Variations are routinely programmed in genetic programming, no genuine creativity required.”
Neil replies:
Programming syntactic variations is easy. The problem is conjecturing which variations are explanatorily relevant given the problem and our background explanatory knowledge. The epistemological work includes conjecturing the relevant search space itself. Otherwise I can generate billions of mutations without discovering whether changing some feature destroys the explanation or merely changes its wording.
That’s why I said it’s fine to prompt the user for creative input if necessary, within reason. #5439
I reply that “prompting users for creative input (within reason) would be fine, as the bounty states.”
Why is that a criticism of the bounty and not of HTV? If that’s what’s required to make HTV work, maybe HTV bit off more than it can chew.
Well, no, people do conjecture variations, and so could use HTV themselves, though maybe not program a computer to do it for them.
Neil Hudson emphasizes that “the procedure by which HTV is implemented cannot itself become the authority that decides which criticisms count. Its criteria, classifications and outputs would themselves have to remain conjectural and open to independent criticism - including criticism not already licensed by the procedure itself. Otherwise HTV has ceased to be a fallible aid to criticism and has become a final epistemic tribunal.”
I reply:
The UI can show a *tentative* result. The result can change with more user input. All input remains criticizable.
The UI can show a *tentative* result. The result can change with more user input. All input remains criticizable. Again, I’m not looking for a criterion of truth or some guarantee of infallibility, which I know can’t exist. I’m looking for a criterion of rationality, which I know can: https://x.com/dchackethal/status/2091272979791659054
The word “again” refers to my previous remarks about fallibility here.
Neil Hudson says, in short, that there’s a difference between a constraint on criticism and a ranking procedure. He says HTV is meant as a constraint, not a ranking procedure.
Like most people, he’s confused about the purpose of HTV. My blog post linked in #3069 already explains.
Neil Hudson asks:
A fallible procedure can still be making the stronger claim that every admissible pair is algorithmically rankable. Why should HTV- or rationality generally - be required to supply that [such an algorithmic ranking]?
I reply:
Because it’s better for error correction. If you want your life to be a beginning of infinity, if you want to make sustained progress, you need to be rational. To be rational, you need to be able to tell when you’re being irrational. So you need a criterion of rationality. The more inputs it works for, the better. Universal is best.
I reply:
Rational decision-making (involving HTV or not) is a computational task. We can’t claim to have understood such a task unless we’ve programmed it.
I reply:
The task would actually be *harder* to program if we constrained it to work only for some ideas and not others.
Neil Hudson says Popper’s criterion of demarcation doesn’t give us a way to form rational preferences. He implies: so why should HTV?
I reply:
The purpose of demarcation was to delineate science from non-science. That’s different from a criterion of rationality.
For such a criterion, Popper proposes measures of corroboration and verisimilitude. And he get pretty technical in defining those.
Have some thoughts, which might be way off. But interested in your response. It seems to me that "hard-to-vary" is itself the criterion that a theory should be as programmable as possible. As you note, the goal of a theory should be to make it as explicit as possible, and a program is explicitness in its most complete form. Any theory with ambiguous components automatically has a breaking point that is changeable which is hard to detect. A programmable theory has strict causal relations all the way from the axioms to the prediction, which makes any change to the components detectable. In other words: a theory is hard to vary to the extent that its components and the couplings between them can be specified as a program. If a theory is vague, you cannot tell when it has been varied.
This might give a concrete operationalization. A breaking point is any place in the formalization where the chain stops being programmable: a primitive with no implementable type, a coupling between components that cannot be turned into a function, or just a step that requires implicit theories to fill the explanatory gaps. A mathematical theory with no remaining gaps has zero breaking points and is maximally hard to vary. A theory in natural language is already worse, because words carry ambiguity and vary from mind to mind. This does not rule out better and worse theories in natural language, since we can use more or less ambiguous words and relations. But it does create a hierarchy of hard-to-vary explanations, where the share of the explanation that is programmable, or at least unambiguous, forms the basis for measuring the "hard-to-vary" criterion.
This is probably too crude a formalization. But evaluating the two theories of Demeter's emotions and axial tilt as explanations, you could check how much of each is programmable. Detecting seasons is programmable in both cases through temperature and changes in weather. Demeter's emotions and the causal link from them to the weather, which is the entire explanation, are not programmable. In the axial tilt theory, every component is. So on this measure Demeter scores 25% and axial tilt scores 100%.
We can redefine ‘hard to vary’, but we’d need still a working implementation in the form of computer code.
… Demeter scores 25% and axial tilt scores 100%.
Now do this universally, for any given theory.
By the way Knut, when I go straight into ‘criticism mode’, that can sound cold or harsh. But don’t let that discourage you from exploring your idea further. Maybe you’re onto something! A working implementation of hard to vary would be useful and vindicating.
If we normalize a theory into the parts that can be put on a computer, the types it uses, the nodes (specific values) it commits to, and the functions between them, we can score the theory by how many of those parts run.
The program goes through each item in a theory, counts how many are marked True, and divides by the total. That fraction is the score of how hard the theory is to vary.
An item is True if it can be put on a computer, either by reusing an existing type (Float, Int) or by defining a new one that compiles. It is False if no working type system can express it. The user fills in the labels; the program just counts.
Demeter: 2 of 8 items program (Latitude and Temperature). Demeter, her emotions, and the functions linking them to weather can't. So the score is 25%.
Axial tilt: 9 of 9 items program. Standard types, measured constants, and functions from standard physics. So the score is 100%.