Activity Feed
#5576·Dennis HackethalOP, 2 days agoOne could run the program again with corrected inputs but as I noted in #5533, that approach won’t scale.
Jad originally replied in #5559. I’ve extracted his reply to avoid bulk criticism:
If the user later changes their mind about one of those judgments, then yes, the current minimal CLI requires them to run it again with the corrected input. I agree that wouldn't be good UX for a full application, but that's not what I'm trying to build here. Adding the ability to add, edit, or delete variations and immediately recalculate the ranking is straightforward and doesn't change the decision procedure being demonstrated.
@jadelmourad be sure to submit separate criticisms separately. You benefit because it means I need to address each criticism individually, not in ‘bulk’. (See #5546, keyword ‘bulk’.)
#5567·Dennis HackethalOP, 2 days agoBut I don't think adding “while I wear one green hat,” “while I wear two green hats,” etc. to the axial-tilt explanation necessarily varies the explanation at all.
If you don’t think those are genuine variations, the program should either reject them or, in light of #5553, it should give users a way to resolve disagreement about what makes a genuine variation, because such a resolution would presumably itself involve some measure of HTV and so is part of the problem of making HTV work.
As the bounty states, human input is fine within reason. It’s beginning to look like your program does relatively little compared to the amount of work it expects its users to do.
I don't think those are genuine variations, so I wouldn't input them when running the program. But someone else might genuinely think they are.
The program can't decide which of us is right without itself understanding the explanation and the problem situation—in other words, without being a general intelligence. That's the point I just made in #5572, and more generally in #5553 and #5557.
At the current stage, without AGI, I think any work that requires that kind of understanding and creative judgment necessarily has to be outsourced to the user. The program then operates on the judgments the user supplies.
#5536·Dennis HackethalOP, 3 days agoThere are still all the normal problems of human knowledge: maybe I missed a variation, maybe I accepted a bad one, maybe someone else would judge things differently. But those same problems exist when we come up with and judge criticisms.
That’s true, but it isn’t clear how criticisms fit into the program as proposed. The BoI chapter 1 glossary (p. 31) defines:
Good/bad explanation An explanation that is hard/easy to vary while
still accounting for what it purports to account for.The program currently has no way to tell, neither automatically nor through human input, whether a variation of an explanation still accounts for what it purports to account for. One could run the program again with corrected inputs but as I noted in #5533, that approach won’t scale.
One could run the program again with corrected inputs but as I noted in #5533, that approach won’t scale.
Jad originally replied in #5559. I’ve extracted his reply to avoid bulk criticism:
If the user later changes their mind about one of those judgments, then yes, the current minimal CLI requires them to run it again with the corrected input. I agree that wouldn't be good UX for a full application, but that's not what I'm trying to build here. Adding the ability to add, edit, or delete variations and immediately recalculate the ranking is straightforward and doesn't change the decision procedure being demonstrated.
Extract out separate criticism to prevent bulk criticism
I think the program already gets this through human input.
The prompt isn't asking the user to enter arbitrary changes. It specifically asks:
“Enter variations of this explanation that would still work to explain the same thing.”
So by entering a variation, the user is making the judgment that it still accounts for what the original explanation was supposed to account for. That's intentionally a human judgment, just like the other judgments I've discussed in #5553.
If the user later changes their mind about one of those judgments, then yes, the current minimal CLI requires them to run it again with the corrected input. I agree that wouldn't be good UX for a full application, but that's not what I'm trying to build here. Adding the ability to add, edit, or delete variations and immediately recalculate the ranking is straightforward and doesn't change the decision procedure being demonstrated.
I think the program already gets this through human input.
The prompt isn't asking the user to enter arbitrary changes. It specifically asks:
“Enter variations of this explanation that would still work to explain the same thing.”
So by entering a variation, the user is making the judgment that it still accounts for what the original explanation was supposed to account for. That's intentionally a human judgment, just like the other judgments I've discussed in #5553.
#5558·Jad Elmourad, 2 days agoI don't think the program needs to choose one of the surviving variants if our current knowledge gives us no reason to choose.
If several variants of the better explanation still work and we currently have nothing that differentiates them, then I think it's fine for them to remain competing possibilities. In fact, my guess is that this is exactly how different research programs can get started: people can follow the different explanatory threads until further criticism, experiments, observations, or other new knowledge gives us a reason to distinguish them.
So I don't think the program should invent a preference between variants when our current problem situation and knowledge don't provide one. Sometimes the rational state really is that we don't yet know which variant is right. We can continue developing and testing them until we find something that differentiates them.
I don't think the program needs to choose one of the surviving variants if our current knowledge gives us no reason to choose.
It does. Preference formation is the point of the program.
Sometimes the rational state really is that we don't yet know which variant is right. We can continue developing and testing them until we find something that differentiates them.
Then we’d need to find a way to feed the results of the tests back into the program. Tests would somehow need to be formally tied to HTV. The program has to at least be able to tell the user which preference to form (short of the user not typing in the variant he’s already decided he doesn’t want, eg because of a negative test case).
#5566·Dennis HackethalOP, 2 days agoI would consider that a problem with the explanation, not with the HTV comparison. It's exactly the kind of arbitrariness that HTV is supposed to expose.
Yes but if the program is an accurate representation/use of HTV, then the program should expose it. The program asks the user for variations in service of exposing it. But that doesn’t work in this case. I think that’s a problem.
I think the program does expose it, given inputs that reflect the user's genuine understanding of the explanations.
If I genuinely regard “axial tilt while I wear one green hat,” “axial tilt while I wear two green hats,” etc. as distinct working variations of the axial-tilt explanation, then I've told the program that the hat count is part of the explanation and can be varied freely. In that case, I think it's completely fair for HTV to judge the explanation easier to vary.
If I instead understand the hat as an irrelevant fact that has nothing to do with the axial-tilt explanation, then I shouldn't submit changes in my hat count as variations of that explanation in the first place.
The program can't accept my judgment that these are genuine variations and then independently override that judgment because it “knows” they're actually irrelevant. Doing that would require the program itself to understand what is doing explanatory work. That's precisely the part I'm outsourcing to the user.
So I don't think this example shows that the program fails to expose easy-to-varyness. It shows that the result depends on which changes the user genuinely judges to be variations of the explanation—which is the same point I made in #5553 and #5555.
#5561·Jad Elmourad, 2 days agoI don't think it follows that a competing explanation would also have uncountably many working variants just because it involves continuous quantities.
The grass example is easy to vary precisely because there's no explanation for why the amount should be 1kg rather than 0.9kg, 0.95kg, etc. If 1kg doesn't work, I can just keep changing the number. Nothing in the explanatory argument constrains it.
But suppose grass really did cure the disease and we actually understood how. Maybe a particular compound in the grass interacts with some biological mechanism, and a certain concentration is required for the effect. The explanation might allow a whole range of effective dosages rather than one exact number, but that range would itself be explained and constrained by the mechanism.
I wouldn't consider every possible dosage within that range a different variation of the explanation. They're different cases covered by the same explanation. Saying “this mechanism works within this dosage range” is one explanatory claim, even if there are infinitely many numerical values inside the range.
So I think there's a difference between an explanation containing a continuous variable and an explanation whose parameters can be changed arbitrarily without explanatory reason. The first can be perfectly constrained by the explanatory structure. The second is exactly what I understand easy-to-vary to mean.
This also comes back to the point I made in #5553: the program relies on the user's understanding of the explanatory argument. A knowledgeable user wouldn't enter 0.9kg, 0.9001kg, 0.90001kg, etc. as infinitely many distinct explanatory variations if they understand them as instances of one explained dosage range.
But suppose grass really did cure the disease and we actually understood how. Maybe a particular compound in the grass interacts with some biological mechanism, and a certain concentration is required for the effect. The explanation might allow a whole range of effective dosages rather than one exact number, but that range would itself be explained and constrained by the mechanism.
That assumes the cure works: then I agree there’s a reason for the specific range, and that range will be part of the one working variant. Your program would then accurately prefer it over a rival.
But the program also needs to account for cases where the cure doesn’t work. Then we can’t refer to a range anymore because there’s no reason to constrain the explanation to that range. And then we’re left with the situation described in #5549.
#5557·Jad Elmourad, 2 days agoI agree that people can disagree about what counts as a variation versus a completely different explanation. I don't agree that the program itself needs to recognize the difference.
The program isn't intended to distinguish a variation from a completely different explanation. That's something the user has to decide. In fact, I don't see how a program could make that distinction in general without being a general intelligence that actually comprehends the problem situation and the explanations involved.
And people themselves can disagree about what counts as a variation. That's unavoidable because it depends on how they understand the underlying explanatory argument. What looks like a harmless change to me might look to someone with a deeper understanding like a change that completely breaks the explanation.
So I don't think different people getting different results is necessarily a problem. If I understand an explanation poorly, I may think lots of its details can be changed independently. Once I understand more of the connections between those details, I may realize that many of those variations don't actually work. On the other hand, an expert might also know valid ways of varying an explanation that I would never have thought of. The result reflects our current understanding, and that can change as our knowledge changes.
This is also part of the broader point I made in #5553: the program operates on the user's current understanding of the problem and explanations rather than independently understanding them for the user.
I don’t think this response addresses the part from #5531 that says “nor is the user given an opportunity to correct any mistakes to that effect.”
#5557·Jad Elmourad, 2 days agoI agree that people can disagree about what counts as a variation versus a completely different explanation. I don't agree that the program itself needs to recognize the difference.
The program isn't intended to distinguish a variation from a completely different explanation. That's something the user has to decide. In fact, I don't see how a program could make that distinction in general without being a general intelligence that actually comprehends the problem situation and the explanations involved.
And people themselves can disagree about what counts as a variation. That's unavoidable because it depends on how they understand the underlying explanatory argument. What looks like a harmless change to me might look to someone with a deeper understanding like a change that completely breaks the explanation.
So I don't think different people getting different results is necessarily a problem. If I understand an explanation poorly, I may think lots of its details can be changed independently. Once I understand more of the connections between those details, I may realize that many of those variations don't actually work. On the other hand, an expert might also know valid ways of varying an explanation that I would never have thought of. The result reflects our current understanding, and that can change as our knowledge changes.
This is also part of the broader point I made in #5553: the program operates on the user's current understanding of the problem and explanations rather than independently understanding them for the user.
So I don't think different people getting different results is necessarily a problem.
#5531 doesn’t say that.
#5556·Jad Elmourad, 2 days agoI think this is where the problem/question given to the program matters.
If the problem I'm trying to answer is “Why does Earth have seasons?”, then replacing Earth with Mars doesn't give me another working variation of the explanation of Earth's seasons. I've changed the phenomenon I'm trying to explain. Applying the same underlying theory to explain seasons on Mars seems to me like an example of its reach, not a variation of its explanation of Earth's seasons.
This is one reason the program asks for the question first. Whether a proposed variation still works can't be judged independently of what we're trying to explain.
As I explained in #5553, the program relies on the user's understanding of both the problem situation and the explanation. So I agree that reach and easy-to-vary shouldn't be confused, but I don't think the program requires us to confuse them. The user has to judge whether they've varied an explanation while preserving what it was supposed to explain, or instead applied the explanation to a different problem.
And of course people can disagree about that distinction, because they can understand the problem situation and explanatory argument differently. That's part of the human judgment the program deliberately leaves to the user.
I think #5567 applies here, too.
#5555·Jad Elmourad, 2 days agoAs I argued in #5554, if the number of green hats is genuinely part of an explanation and can be changed arbitrarily without affecting anything, then I think that really does expose an easy-to-vary part of the explanation.
But I don't think adding “while I wear one green hat,” “while I wear two green hats,” etc. to the axial-tilt explanation necessarily varies the explanation at all.
If my explanation is that Earth's seasons result from its axial tilt and the resulting relationship between Earth and the Sun, whether I happen to be wearing one green hat or a thousand isn't part of that explanatory argument. Appending an unrelated fact to the English sentence doesn't change the explanation it expresses.
If, on the other hand, I genuinely claim that my hat is part of the explanation—say, that Earth's seasons occur because of axial tilt and because I'm wearing exactly one green hat—then changing the number of hats would be a genuine variation. But in that case I really have made the explanation worse by adding an arbitrary component that isn't constrained by what I'm trying to explain.
So I think there's a difference between varying an explanation and varying the string of English used to express it. The program relies on the user to understand that distinction, for the same reason I described in #5553: it operates on the user's understanding of the explanatory argument, not on arbitrary syntactic changes to its wording.
On that understanding, I don't think both explanations become equally easy to vary merely because infinitely many irrelevant statements can be appended to the sentences expressing them.
But I don't think adding “while I wear one green hat,” “while I wear two green hats,” etc. to the axial-tilt explanation necessarily varies the explanation at all.
If you don’t think those are genuine variations, the program should either reject them or, in light of #5553, it should give users a way to resolve disagreement about what makes a genuine variation, because such a resolution would presumably itself involve some measure of HTV and so is part of the problem of making HTV work.
As the bounty states, human input is fine within reason. It’s beginning to look like your program does relatively little compared to the amount of work it expects its users to do.
#5554·Jad Elmourad, 2 days agoAs I explained in #5553, the program operates on the user's current understanding of the explanation.
In this case, though, I think the green-hat example actually illustrates what it means for an explanation to be easy to vary.
Suppose my explanation really is “God causes the seasons while wearing one green hat,” and I genuinely think I can change that to two hats, three hats, a thousand hats, etc. without affecting the explanation at all. Then the number of hats is completely unconstrained by what I'm trying to explain. Nothing in the explanation tells me why it should be one rather than two or a thousand.
I would consider that a problem with the explanation, not with the HTV comparison. It's exactly the kind of arbitrariness that HTV is supposed to expose.
Of course, the program isn't going to enumerate infinitely many hats. It only works with the variations the user actually comes up with. But if I can keep producing arbitrary variations without affecting the explanatory argument, that seems like evidence that the explanation is easy to vary, not a reason to reject the method.
I would consider that a problem with the explanation, not with the HTV comparison. It's exactly the kind of arbitrariness that HTV is supposed to expose.
Yes but if the program is an accurate representation/use of HTV, then the program should expose it. The program asks the user for variations in service of exposing it. But that doesn’t work in this case. I think that’s a problem.
#5525·Dennis HackethalOP, 3 days agoTake the question: Why do seasons occur?
Should credit Deutsch for the example. Reference BoI chapter 1.
Agreed. Revised in #5551.
#5524·Dennis HackethalOP, 3 days agoShe spends part of the year in Hades…
Hades is god the underworld, not a place.
Agreed. Revised in #5551.
#5543·Dennis HackethalOP, 3 days agoThere’s currently no validation, neither automatic nor creative. A user could input junk and there’s no way to correct that short of running the program a second time. (The revision may already address this criticism, I haven’t checked it yet.)
Same point as my response in #5559.
#5538·Dennis HackethalOP, 3 days agoThere are still all the normal problems of human knowledge: maybe I missed a variation, maybe I accepted a bad one, maybe someone else would judge things differently. But those same problems exist when we come up with and judge criticisms.
In this context, there’s a serious issue at the heart of HTV in that not all preference formation involves thinking about variants. Oftentimes, preference formation simply doesn’t take this form. Successful rehabilitation of HTV would be universal in this sense: it would reduce all preference formation to HTV somehow.
I’ve given the example of Pop Tarts in the past. You can ask someone why they’re called that. And they might guess some answers (‘because they “pop” out of the toaster’, ‘because the name is fun’, ‘because the filling tastes tart’, etc.). But ultimately the preferable (and correct) answer will be gotten by just looking it up on the company website, say. Not by seeing which guess is “harder to vary”. I think that should count toward the “normal problems” you mention, but it’s one that needs to be addressed.
plaintext➜ hard-to-vary git:(master) python3 htv.pyHard-to-Vary Explanation Comparator-----------------------------------What question are you trying to answer?> Why are they called Pop Tarts?Enter explanations one at a time.Press Enter on an empty line when you're done.Explanation 1: Because they ‘pop’ out of the toaster.Explanation 2: Because the name is fun.Explanation 3: Because the filling tastes tart.Explanation 4:============================================================Explanation:Because they ‘pop’ out of the toaster.Enter variations of this explanation that would still work to explain the same thing.Press Enter on an empty line when you're done.Variation 1:============================================================Explanation:Because the name is fun.Enter variations of this explanation that would still work to explain the same thing.Press Enter on an empty line when you're done.Variation 1:============================================================Explanation:Because the filling tastes tart.Enter variations of this explanation that would still work to explain the same thing.Press Enter on an empty line when you're done.Variation 1:============================================================Question:Why are they called Pop Tarts?============================================================HARDNESS-TO-VARY RANKING============================================================Rank 1Explanation: Because they ‘pop’ out of the toaster.Working variations submitted: 0 variationsRank 1Explanation: Because the name is fun.Working variations submitted: 0 variationsRank 1Explanation: Because the filling tastes tart.Working variations submitted: 0 variationsFewer working variations = harder to vary.In this case, I couldn’t think of any good-faith variants of any of the three guesses. So they end up being equally preferable according to the program. But I happen to know that all three are false.
If I didn’t know, I’d reject guess 3 because Pop Tarts don’t actually taste tart. They taste sweet. So that’s a pending criticism. But guess 1 and 2 are plausible and I see no reason to prefer one over the other. One would need additional info, but not variant guesses.
I think the Pop-Tarts example mixes together two different things: acquiring new knowledge and forming a preference between explanations. Looking up why Pop-Tarts were actually given their name gives us new knowledge; it doesn't by itself show that the preference formation between explanations isn't HTV.
As I argued in #5553, the program operates on the user's current understanding of the problem and explanations. The user is comparing explanations and variations they currently judge to work. So everything they already know, including criticisms they're aware of, is part of that judgment.
In the Pop-Tarts example, say the three explanations are equally hard to vary given what I currently know. That's fine. I currently have no reason to prefer one over another.
Now suppose I look it up and find historical evidence saying the name was chosen for reason X. I've gained new knowledge, and that changes the problem situation. I'm no longer just asking why they're called Pop-Tarts; I'm asking why they're called Pop-Tarts given that I also know this historical evidence says X.
I could still maintain that the real reason was something else. Maybe the company was lying or the historical source was wrong. But then I have to explain that too. If I just say “the real reason is that they pop out of the toaster, and for some unrelated reason the company says X,” I've added another arbitrary factor to preserve my explanation. That seems like exactly the kind of thing that would expose it as easy to vary.
So I don't see acquiring new knowledge as something that sits outside the process and simply overrides the HTV result. New knowledge changes the problem situation and therefore changes what a working explanation now has to account for. We can then compare the explanations again in light of that new problem situation.
Being hard to vary also doesn't guarantee that an explanation is true. Newton's theory of gravity was a good explanation and remains good enough for many engineering problem situations, even though we later learned that it isn't the full story. As new problems and observations came up that Newton couldn't adequately account for, the problem situation changed and we needed a better explanation.
So I don't think every act of acquiring knowledge or finding a criticism itself needs to take the form of thinking about variants. Those things change the problem situation and affect which explanations and variations we consider to still work.
But that's different from saying the preference formation itself isn't HTV. Once our current knowledge and criticisms are taken into account, HTV can still operate on the explanations that still work and give us a preference—or sometimes tell us that, given what we currently know, they're equally good.
#5549·Dennis HackethalOP, 3 days agoAs I recall, FoR has an example of a grass cure. It says something like: you could eat 1g of grass to cure some disease. If that doesn’t work, proponents of that theory can simply say you actually need to eat 0.9g. When that doesn’t work either, they can say you need 0.95g. And so on. So there is literally an uncountably infinite number of variant theories: one for each decimal number in that vicinity.
Put those variants up against a competing explanation that also has uncountably many variants. Now you can arbitrarily get a result saying to prefer one over the other by typing in fewer variants for one than for the other.
More abstractly put, we can’t always count the variants. But the algorithm, as written, expects us to.
I don't think it follows that a competing explanation would also have uncountably many working variants just because it involves continuous quantities.
The grass example is easy to vary precisely because there's no explanation for why the amount should be 1kg rather than 0.9kg, 0.95kg, etc. If 1kg doesn't work, I can just keep changing the number. Nothing in the explanatory argument constrains it.
But suppose grass really did cure the disease and we actually understood how. Maybe a particular compound in the grass interacts with some biological mechanism, and a certain concentration is required for the effect. The explanation might allow a whole range of effective dosages rather than one exact number, but that range would itself be explained and constrained by the mechanism.
I wouldn't consider every possible dosage within that range a different variation of the explanation. They're different cases covered by the same explanation. Saying “this mechanism works within this dosage range” is one explanatory claim, even if there are infinitely many numerical values inside the range.
So I think there's a difference between an explanation containing a continuous variable and an explanation whose parameters can be changed arbitrarily without explanatory reason. The first can be perfectly constrained by the explanatory structure. The second is exactly what I understand easy-to-vary to mean.
This also comes back to the point I made in #5553: the program relies on the user's understanding of the explanatory argument. A knowledgeable user wouldn't enter 0.9kg, 0.9001kg, 0.90001kg, etc. as infinitely many distinct explanatory variations if they understand them as instances of one explained dosage range.
#5537·Dennis HackethalOP, 3 days agoI don't think either program solves those problems, and I don't think it needs to. That's the part people do.
I think Veritula solves it because it has a recursive notion of criticism. It doesn’t ‘solve’ it in the sense that it completely takes all burden off humans – creative input is still required – but it solves it in the sense that new input can be given at runtime that changes the displayed (tentative) result.
In any case, although the blog post suggests Veritula as a working alternative to HTV, the bounty isn’t a competition between HTV and Veritula. It’s about rehabilitating HTV in its own right. That may require addressing flaws even if Veritula has them, too.
I agree that HTV has to stand on its own. My argument isn't “Veritula has this problem too, therefore it's okay for HTV to have it.”
The reason I've been comparing the two is narrower: to show that outsourcing the parts requiring creativity and judgment to the user doesn't prevent us from having an executable decision procedure built on top of those inputs.
On revisability specifically, I agree that the result should be tentative and able to change when the user's judgments change. As I explained in #5559, this can technically already be done by rerunning the program with the updated inputs. The current CLI is only a minimal demonstration, so that's obviously not ideal UX. Letting the user add, edit, or delete variations and immediately recalculate the result would be straightforward and wouldn't change the decision procedure.
#5536·Dennis HackethalOP, 3 days agoThere are still all the normal problems of human knowledge: maybe I missed a variation, maybe I accepted a bad one, maybe someone else would judge things differently. But those same problems exist when we come up with and judge criticisms.
That’s true, but it isn’t clear how criticisms fit into the program as proposed. The BoI chapter 1 glossary (p. 31) defines:
Good/bad explanation An explanation that is hard/easy to vary while
still accounting for what it purports to account for.The program currently has no way to tell, neither automatically nor through human input, whether a variation of an explanation still accounts for what it purports to account for. One could run the program again with corrected inputs but as I noted in #5533, that approach won’t scale.
I think the program already gets this through human input.
The prompt isn't asking the user to enter arbitrary changes. It specifically asks:
“Enter variations of this explanation that would still work to explain the same thing.”
So by entering a variation, the user is making the judgment that it still accounts for what the original explanation was supposed to account for. That's intentionally a human judgment, just like the other judgments I've discussed in #5553.
If the user later changes their mind about one of those judgments, then yes, the current minimal CLI requires them to run it again with the corrected input. I agree that wouldn't be good UX for a full application, but that's not what I'm trying to build here. Adding the ability to add, edit, or delete variations and immediately recalculate the ranking is straightforward and doesn't change the decision procedure being demonstrated.
#5534·Dennis HackethalOP revised 3 days agoWhen variations are given for both competing explanations, the user is none the wiser which variation of the ‘better’ explanation he should adopt. There’s no clear preference:
Whenever a wide range of variant theories can account equally well for the phenomenon they are trying to explain, there is no reason to prefer one of them over the others, so advocating a particular one in preference to the others is irrational.
#5526 is an example. Even ignoring the obviously false outcome, which I caused for illustrative purposes, we don’t know which variation of the better explanation to prefer.
I don't think the program needs to choose one of the surviving variants if our current knowledge gives us no reason to choose.
If several variants of the better explanation still work and we currently have nothing that differentiates them, then I think it's fine for them to remain competing possibilities. In fact, my guess is that this is exactly how different research programs can get started: people can follow the different explanatory threads until further criticism, experiments, observations, or other new knowledge gives us a reason to distinguish them.
So I don't think the program should invent a preference between variants when our current problem situation and knowledge don't provide one. Sometimes the rational state really is that we don't yet know which variant is right. We can continue developing and testing them until we find something that differentiates them.
#5531·Dennis HackethalOP, 3 days ago@dirk-meulenbelt says it isn’t clear what counts as a variation vs an entirely different explanation. I’ll add that the program can’t currently recognize the difference, nor is the user given an opportunity to correct any mistakes to that effect.
I agree that people can disagree about what counts as a variation versus a completely different explanation. I don't agree that the program itself needs to recognize the difference.
The program isn't intended to distinguish a variation from a completely different explanation. That's something the user has to decide. In fact, I don't see how a program could make that distinction in general without being a general intelligence that actually comprehends the problem situation and the explanations involved.
And people themselves can disagree about what counts as a variation. That's unavoidable because it depends on how they understand the underlying explanatory argument. What looks like a harmless change to me might look to someone with a deeper understanding like a change that completely breaks the explanation.
So I don't think different people getting different results is necessarily a problem. If I understand an explanation poorly, I may think lots of its details can be changed independently. Once I understand more of the connections between those details, I may realize that many of those variations don't actually work. On the other hand, an expert might also know valid ways of varying an explanation that I would never have thought of. The result reflects our current understanding, and that can change as our knowledge changes.
This is also part of the broader point I made in #5553: the program operates on the user's current understanding of the problem and explanations rather than independently understanding them for the user.
#5530·Dennis HackethalOP, 3 days agoSome variations are good (and then described as reach). I could vary the axis-tilt explanation of the seasons by replacing earth with any other tilted planet (as I did in #5526). I could do this infinitely many times, including for yet-to-be-discovered planets and even imaginary ones. But that doesn’t make the explanation worse. (That isn’t your fault, I think Deutsch just hasn’t gotten technical enough about the difference between reach vs. easy to vary.)
I think this is where the problem/question given to the program matters.
If the problem I'm trying to answer is “Why does Earth have seasons?”, then replacing Earth with Mars doesn't give me another working variation of the explanation of Earth's seasons. I've changed the phenomenon I'm trying to explain. Applying the same underlying theory to explain seasons on Mars seems to me like an example of its reach, not a variation of its explanation of Earth's seasons.
This is one reason the program asks for the question first. Whether a proposed variation still works can't be judged independently of what we're trying to explain.
As I explained in #5553, the program relies on the user's understanding of both the problem situation and the explanation. So I agree that reach and easy-to-vary shouldn't be confused, but I don't think the program requires us to confuse them. The user has to judge whether they've varied an explanation while preserving what it was supposed to explain, or instead applied the explanation to a different problem.
And of course people can disagree about that distinction, because they can understand the problem situation and explanatory argument differently. That's part of the human judgment the program deliberately leaves to the user.
#5529·Dennis HackethalOP, 3 days agoIn a situation where both competing explanations have infinitely many variations (eg #5528), the algorithm returns an equal ranking, even though one explanation is clearly preferable over the other. (“Because of axial tilt while I wear 1 green hat”, though still bad, is intuitively preferable to “Because god did it while wearing 1 green hat.”)
As I argued in #5554, if the number of green hats is genuinely part of an explanation and can be changed arbitrarily without affecting anything, then I think that really does expose an easy-to-vary part of the explanation.
But I don't think adding “while I wear one green hat,” “while I wear two green hats,” etc. to the axial-tilt explanation necessarily varies the explanation at all.
If my explanation is that Earth's seasons result from its axial tilt and the resulting relationship between Earth and the Sun, whether I happen to be wearing one green hat or a thousand isn't part of that explanatory argument. Appending an unrelated fact to the English sentence doesn't change the explanation it expresses.
If, on the other hand, I genuinely claim that my hat is part of the explanation—say, that Earth's seasons occur because of axial tilt and because I'm wearing exactly one green hat—then changing the number of hats would be a genuine variation. But in that case I really have made the explanation worse by adding an arbitrary component that isn't constrained by what I'm trying to explain.
So I think there's a difference between varying an explanation and varying the string of English used to express it. The program relies on the user to understand that distinction, for the same reason I described in #5553: it operates on the user's understanding of the explanatory argument, not on arbitrary syntactic changes to its wording.
On that understanding, I don't think both explanations become equally easy to vary merely because infinitely many irrelevant statements can be appended to the sentences expressing them.
#5528·Dennis HackethalOP, 3 days agoThe approach underestimates how easy it is to come up with variations. It’s trivially easy to come up with infinitely many just by, for example, adding nonsense that has an incrementing counter:
plaintext➜ hard-to-vary git:(master) python3 htv.pyHard-to-Vary Explanation Comparator-----------------------------------What question are you trying to answer?> Why are there seasons?Enter explanations one at a time.Press Enter on an empty line when you're done.Explanation 1: Because god did it.Explanation 2: Because of axial tilt.Explanation 3:============================================================Explanation:Because god did it.Enter variations of this explanation that would still work to explain the same thing.Press Enter on an empty line when you're done.Variation 1: Because god did it while wearing 1 green hat.Variation 2: Because god did it while wearing 2 green hats.Variation 3: Because god did it while wearing 3 green hats.Variation 4: Because god did it while wearing 4 green hats. And so on.Variation 5:============================================================Explanation:Because of axial tilt.Enter variations of this explanation that would still work to explain the same thing.Press Enter on an empty line when you're done.Variation 1: Because of axial tilt while I wear 1 green hat.Variation 2: Because of axial tilt while I wear 2 green hats.Variation 3: Because of axial tilt while I wear 3 green hats.Variation 4: Because of axial tilt while I wear 4 green hats. And so on.Variation 5:============================================================Question:Why are there seasons?============================================================HARDNESS-TO-VARY RANKING============================================================Rank 1Explanation: Because god did it.Working variations submitted: 4 variationsRank 1Explanation: Because of axial tilt.Working variations submitted: 4 variationsFewer working variations = harder to vary.Result:The two explanations are equally hard to vary.
As I explained in #5553, the program operates on the user's current understanding of the explanation.
In this case, though, I think the green-hat example actually illustrates what it means for an explanation to be easy to vary.
Suppose my explanation really is “God causes the seasons while wearing one green hat,” and I genuinely think I can change that to two hats, three hats, a thousand hats, etc. without affecting the explanation at all. Then the number of hats is completely unconstrained by what I'm trying to explain. Nothing in the explanation tells me why it should be one rather than two or a thousand.
I would consider that a problem with the explanation, not with the HTV comparison. It's exactly the kind of arbitrariness that HTV is supposed to expose.
Of course, the program isn't going to enumerate infinitely many hats. It only works with the variations the user actually comes up with. But if I can keep producing arbitrary variations without affecting the explanatory argument, that seems like evidence that the explanation is easy to vary, not a reason to reject the method.