Hard to Vary or Hardly Usable?
Showing only ideas leading to #5631.
See full discussion·See most recent related ideasLog in or sign up to participate in this discussion.
With an account, you can revise, criticize, and comment on ideas.My critique of David Deutsch’s The Beginning of Infinity as a programmer. In short, his ‘hard to vary’ criterion at the core of his epistemology is fatally underspecified and impossible to apply.
Deutsch says that one should adopt explanations based on how hard they are to change without impacting their ability to explain what they claim to explain. The hardest-to-change explanation is the best and should be adopted. But he doesn’t say how to figure out which is hardest to change.
A decision-making method is a computational task. He says you haven’t understood a computational task if you can’t program it. He can’t program the steps for finding out how ‘hard to vary’ an explanation is, if only because those steps are underspecified. There are too many open questions.
So by his own yardstick, he hasn’t understood his epistemology.
You will find that and many more criticisms here: https://blog.dennishackethal.com/posts/hard-to-vary-or-hardly-usable
A simple implementation of hard-to-vary
This is my submission for Veritula's bounty for Idea #3069, which asks for an executable implementation that can compare arbitrary English explanations by how hard they are to vary.
Code: hard-to-vary/htv.py at 9da0ddd
I've implemented the approach as a small interactive program. It takes arbitrary explanations supplied by the user, collects working variations of each, and ranks the explanations by hardness to vary.
I think hard-to-vary can be implemented more simply than the approaches considered in Dennis's blog, Hard to Vary or Hardly Usable?. The basic idea I'm using is this: if an explanation is harder to vary, there should be fewer ways of changing it while still having it explain what it's supposed to explain.
The program starts by asking the user what question they're trying to answer and then asks for a list of proposed explanations. It goes through each explanation one at a time and asks the user to come up with variations of it that would still work as explanations. The user can enter as many as they can find, then move on to the next explanation.
Variations can either be entered individually or parameterized using notation like <X> and <Y>. For example, <X> kg of grass mixed with <Y> kg of wheat cures the disease can represent a whole family of variations. The user can either enter how many working values or combinations the parameters have, including an infinite number, or enumerate them and let the program count. This avoids requiring the user to manually enter every member of a family of variations.
The user can also review and revise their inputs before the final ranking by adding, editing, or deleting explanations and variations.
Finally, the program counts the working variations and ranks the explanations. Fewer working variations means harder to vary, while equal numbers mean equally hard to vary. So if explanation A has two working variations and explanation B has five, A is harder to vary than B.
Infinite variations need a little more information. If two explanations both have infinitely many working variations, the program first compares how many independently variable parameters have infinitely many possible values. Fewer infinitely-variable parameters means harder to vary. If that's also equal, it compares the remaining finite parts of the variations. This lets the program distinguish, for example, between an explanation with one unconstrained infinite parameter and another with two.
An example
I'll use Deutsch's example from Chapter 1, “The Reach of Explanations,” of The Beginning of Infinity: Why do seasons occur?
One explanation is the story of Persephone. Hades, god of the underworld, kidnaps Persephone. Her mother Demeter eventually negotiates her release under an arrangement that requires Persephone to return to Hades once a year. Whenever Persephone is away, Demeter becomes sad and makes the world cold and bleak.
Deutsch's point is that we can change many of the details of this explanation while still accounting for the same observations. For example, I might try variations like:
- Persephone escapes instead of being released under an agreement.
- Something other than a magic seed compels her to return.
- Some other arrangement causes Persephone to return annually instead of a marriage contract.
The details have changed, but the story can still be made to explain the seasonal cycle. Suppose I find three such variations.
Now I try the explanation involving Earth's axial tilt. Here the details are much more constrained by what we're trying to explain. Changing the geometry substantially changes what the theory predicts about the seasons. Suppose I only find one working variation.
The program therefore ranks axial tilt as harder to vary: one working variation versus three. That's the entire comparison procedure.
How I understand Dennis's criticism
I don't understand Dennis as claiming that Deutsch simply gets the Persephone example wrong. The issue in the blog is how we get from examples like this, where we seem to have an intuition that one explanation is harder to vary, to a sufficiently specified procedure that could compare explanations in general.
Dennis initially explores numerical quality scores, but that immediately creates problems. Why should one explanation have a score of 500 rather than 550? Why choose that scale? How do criticisms affect the score? How do criticisms of criticisms affect it? I agree with Dennis that these choices look arbitrary.
He eventually gets rid of the quality scores entirely. Instead of trying to measure the quality of an idea, his program keeps track of pending criticisms. His proposed rule becomes: adopt ideas without pending criticisms and reject ideas that have them.
There's something important about how that system works, though. The program doesn't generate criticisms itself; people do. And that's intentional: Dennis says "I’m not looking to formalize or automate creativity as a whole". Creative input can come from users while the program handles the non-creative part of the process. I agree that you can have a rational decision-making process while outsourcing the creative part to the user.
I just don't see why we can't do the same thing with HTV. Let the user come up with variations and say which ones they think still work. The program doesn't need to understand the explanation or come up with the variations itself. It keeps track of the variations and compares the counts.
This doesn't seem fundamentally different from Dennis letting the user tell his program that something is a criticism. In fact, in the blog he explicitly avoids having the program figure out whether a comment is really a criticism: the user checks a box saying that it is. So in my program the user is supplying a working variation; in his, the user is supplying a criticism. In both cases the user is providing the part that requires understanding and judgment, and the program does something simple with that input.
It also means I don't need the quality scores Dennis runs into trouble with. I don't need to decide how many points axial tilt gets compared with Persephone. I'm just asking: how many ways have we actually found to change each explanation while still having it work?
But isn't this subjective?
One criticism Dennis quotes is:
“Also, isn’t the difficulty of changing an explanation at least partly a property not of the explanation itself but of whoever is trying to change it? If I’m having difficulty changing it, maybe that’s because I lack imagination. Or maybe I’m just new to that field and an expert could easily change it.”
I think this is true. I just don't think it's a problem specific to HTV.
Imagine I can't think of any working variations of an explanation, but an expert can immediately think of five. Then yes, our results will be different. But isn't that also what happens with criticism? I might look at an idea and fail to see anything wrong with it while someone who knows much more about the subject immediately sees a serious criticism.
The same goes for participation. An idea on Veritula might have zero pending criticisms simply because hardly anyone has tried to criticize it. That doesn't mean there are literally no criticisms of it. Someone could find one tomorrow. Dennis's answer in the blog is basically that if this bothers you, try to find a criticism yourself. If you can't find one, why not adopt the idea?
I think HTV can work the same way. Zero working variations doesn't mean that we've somehow proven there are no possible variations. It means we haven't found one. If you think the explanation is actually easy to vary, try to come up with a variation that still works.
So yes, the result depends on the knowledge and creativity of the person using the program. But I think rational decision-making is always going to depend on what criticisms, arguments, alternatives, etc. a person is actually aware of. I don't see how Veritula escapes that either.
What about human judgment?
There's another obvious question: who decides whether a variation actually works?
For this program, the user does. I don't think we can get rid of that kind of human judgment, at least until we get AGI. Two people can disagree about whether a variation still explains the thing we're trying to explain. They can also disagree about whether two variations are really different or are basically the same variation stated twice.
Again, I think Veritula has the same underlying issue. People still have to decide whether something really is a criticism, whether a countercriticism actually answers it, whether two criticisms are redundant, and so on. Dennis's program can keep track of the structure, but the structure only means something if those judgments make sense.
Take an extreme case. If a malicious moderator rejects every good criticism of an idea and accepts nonsense countercriticisms, the idea could end up showing 0 pending criticisms. I obviously shouldn't look at the 0 and conclude that the idea is rational to adopt. I'd want to read what happened and decide whether I agree with it.
I don't mean this as a criticism specific to Veritula. I think it's just a limit of this kind of approach. At some point people have to make judgments, and people can disagree about them. Ultimately everyone is their own moderator when it comes to their own decision-making. I have to decide which arguments I accept, which criticisms I think have been answered, and so on.
The same is true with my HTV program. If someone gives me a ranking based on ten supposed working variations, I don't have to accept the ranking blindly. I can look at the ten variations and decide that five don't really work and three others are basically duplicates. My result would then be different.
I'm fine with that. I don't think the goal of either program should be to somehow remove judgment from rational thinking. The program gives us a procedure for what to do with the judgments we've made.
What does the program actually contribute?
Dennis writes:
“We can’t just outsource everything to the user – the app has to do some things or it has no value.”
This was actually the part of the blog that made me think about Veritula itself. What is Veritula doing, and what is it outsourcing?
It outsources the interesting creative part to people. People come up with the ideas. People come up with the criticisms. The user can even tell the program whether something they've written is a criticism by checking a box. The program then keeps track of the structure and tells us how many criticisms are pending.
My program is doing something similar, except with variations. People come up with the explanations and the working variations. The program keeps track of them, counts them, and ranks the explanations.
So I don't think I'm outsourcing everything to the user any more than Veritula is. I'm outsourcing the part that requires creativity and judgment. The actual decision rule is implemented in the program.
For Veritula, that rule ultimately depends on whether there are pending criticisms. For my program, it depends on the number of working variations: fewer working variations means harder to vary.
Where I disagree with the blog
I agree with Dennis that the quality sliders in the blog don't work. I also agree with his decision to let users supply the creative input rather than expecting the program to generate it.
Where I disagree is that I think once we allow this same freedom for HTV, we can construct a similarly simple program for it. The user comes up with explanations and tries to vary them while keeping them working. The program counts the working variations and ranks the explanations.
There are still all the normal problems of human knowledge: maybe I missed a variation, maybe I accepted a bad one, maybe someone else would judge things differently. But those same problems exist when we come up with and judge criticisms.
I don't think either program solves those problems, and I don't think it needs to. That's the part people do.
Unfortunately, Jad has admitted to using AI for his submissions, in violation of the bounty terms. As a result, he’s not eligible for a payout. After discussing the issue with him and separately with cofunders @dirk-meulenbelt and @davies, I’ve decided to withdraw my funds.
However, the co-funders say they don’t mind the use of AI. For that reason, and as a token of appreciation for Jad’s idea, I’m making an exception and he’ll remain eligible for their funds, pending further development of this bounty.
Just to confirm, and so you can hear it from me, yes I did use AI to help with my submission.
For posterity, here was my position during the discussion:
I think it's technically fair for Dennis to consider the submission ineligible because the bounty terms clearly say "Submissions must be handwritten. No AI-generated text or code; suspected AI use makes submissions ineligible."
On the other hand, I think this outcome is a shame because I don't think my use of AI changes any of the real work and thought that was put into it. If you zoom out, the motive of the bounty is to attract people to work on ideas and develop them, which is exactly what I did here, regardless of the tools used. As How Do Bounties Work? puts it, "Bounties let you invite criticism and reward high-quality contributions with real money."
And this is exactly what happened here. The idea had been posted on Veritula for about 10 months, and when I saw the bounty, I took it as a fun challenge to work on. So in that sense, the bounty incentive worked. And it seems to me that the reception of the idea has been good so far, which is why I find this outcome disappointing. On a symbolic level—and I acknowledge that this may be very different from how Dennis sees or intends the decision—his withdrawal of all his funds feels to me like this work is somehow not something he's willing to reward.
I did attempt to persuade Dennis to reinterpret the no-AI rule by discussing the underlying reasons for the rule. We agreed that handwriting can be a way to test the submitter's own understanding rather than relying on AI's understanding. While we agreed that this wasn't an issue for me, we disagreed about what should follow from that. My view was that the purpose of the rule should matter more than the literal wording, whereas Dennis seemed to think that the written terms should still govern.
Ultimately, we couldn't come to an agreement on whether my use of AI was appropriate in the context of this bounty. We also disagreed about whether I should have raised my use of AI proactively. In any case, I respect Dennis's decision since ultimately these were the terms of his bounty. We also agreed that in the future, discussions like this about the terms ought to be brought up at the beginning rather than after the fact.
When thousands of dollars are at stake, I think it’s fair to ask people to play by the rules. Still, I’m allowing flexibility by giving Jad a chance to collect the remaining funds.
I’m always happy to reward good work; I’ve paid out bounties in the past. And before I found out about Jad’s use of AI, I gave him tips increasing his odds of beating the bounty. I also submitted criticisms of some of my own ideas, further increasing his odds.
But people need to earn the reward. At some point, I noticed Jad had submitted his replies very quickly, with unusually verbose text, and in fairly rapid succession. AI is notoriously verbose, so that would have been the second-best time for him to mention he was using AI. He did not. But some people do genuinely type fast, so I gave him the benefit of the doubt.
When I later asked him directly whether he was using AI, he admitted to it but claimed it was merely to “help edit some of [his] writings”.
Pangram, an AI detector, paints a different picture: it says 100% (!) of the code and 22% of the prose in his first idea are AI-generated. I’ve also sampled some of his replies since, which Pangram says are also 100% AI-generated.
To play devil’s advocate once again, I wanted to make sure Pangram wasn’t giving me false positives. So I ran several tests against some of my own prose and code, and compared the results with known AI-generated prose and code. Pangram was almost always right – the few times it did make mistakes, they were almost always false negatives, so if anything, Pangram is too permissive. It does have lower confidence for shorter texts, so I also made sure to use only texts that are long enough. I showed Jad these results privately and he didn’t have a good answer, so I’m surprised he now wants to continue the discussion publicly.
In terms of content, handwriting the code matters because of Deutsch’s yardstick for having understood a computational task: if you can’t program it, you haven’t understood it. Jad says he still understands the code regardless, so it shouldn’t matter. But why not raise that before agreeing to a rule he took issue with? Why accept the risk of disqualification instead?
In terms of fairness, as Jad knows, payout for a bounty depends on addressing all known criticisms inside a review deadline. When a participant uses AI to help with that, but the funders don’t know about it, it creates an unfair advantage. That’s a violation of not just the letter but also the spirit of the rules. It’d be like secretly using a chess computer for help in a chess competition. The amount of text Jad has submitted also creates an uphill battle for the remaining funders to review his submissions before the deadline. Jad says he knows how bounties work, so he knows payouts are automatic at the deadline. Still, the remaining funders are willing to accommodate him.
Before I made the decision to withdraw my funds, I reached out to Jad privately, as I said, to make sure there wasn’t anything I had missed and that there were no hard feelings. We discussed the issue thoroughly and I couldn’t find anything. I also discussed the matter with the co-funders for a second and third opinion, again to make sure I hadn’t left any stones unturned. (To be clear, the responsibility for the decision to withdraw my funds is mine alone.)
I understand Jad was hoping for a bigger payout, so this must be a disappointing outcome. I empathize with that. I’d be bummed too. But he could have easily avoided this outcome. One can’t agree to a rule, immediately break it, get an unfair advantage, hope I won’t notice, and then complain when I do. But with any luck, he’ll get the remaining funds, provided that the arguing ends now.
But why not raise that before agreeing to a rule he took issue with? Why accept the risk of disqualification instead?
This is a fair point, like I conceded in #5613: "We also agreed that in the future, discussions like this about the terms ought to be brought up at the beginning rather than after the fact."
And I definitely learned a lesson: to hash things out before if I see any potential conflict.
If you're curious for some more details about what happened here, I would say I got excited about getting my idea out as fast as possible and hearing criticisms. I had noticed the potential conflict about AI use, and I had marked it in my head as something we might have to discuss eventually, but I didn't see that point as arriving until at least the submission seemed to be surviving the criticisms. So when you brought it up and asked whether I was using AI, I immediately said yes.
Veritula discussion trees can’t accurately reflect problematic vs unproblematic ideas when unreasonable people are involved, see #5624. Submitting this as a fix. Jad agrees to terms, breaks them, then wants to argue the purpose and meaning of them. That’s arbitrary, ie the opposite of reason and Veritula. He will no doubt claim he was just trying to follow the Veritula method of addressing criticisms, but the effect is that he’s turning Veritula on its head in the process. There’s no point discussing with people like him.