Activity

When thousands of dollars are at stake, I think it’s fair to ask people to play by the rules. Still, I’m allowing flexibility by giving Jad a chance to collect the remaining funds.

I’m always happy to reward good work; I’ve paid out bounties in the past. And before I found out about Jad’s use of AI, I gave him tips increasing his odds of beating the bounty. I also submitted criticisms of some of my own ideas, further increasing his odds.

But people need to earn the reward. At some point, I noticed Jad had submitted his replies very quickly, with unusually verbose text, and in fairly rapid succession. AI is notoriously verbose, so that would have been the second-best time for him to mention he was using AI. He did not. But some people do genuinely type fast, so I gave him the benefit of the doubt.

When I later asked him directly whether he was using AI, he admitted to it but claimed it was merely to “help edit some of [his] writings”.

Pangram, an AI detector, paints a different picture: it says 100% (!) of the code and 22% of the prose in his first idea are AI-generated. I’ve also sampled some of his replies since, which Pangram says are also 100% AI-generated.

To play devil’s advocate once again, I wanted to make sure Pangram wasn’t giving me false positives. So I ran several tests against some of my own prose and code, and compared the results with known AI-generated prose and code. Pangram was almost always right – the few times it did make mistakes, they were almost always false negatives, so if anything, Pangram is too permissive. It does have lower confidence for shorter texts, so I also made sure to use only texts that are long enough. I showed Jad these results privately and he didn’t have a good answer, so I’m surprised he now wants to continue the discussion publicly.

In terms of content, handwriting the code matters because of Deutsch’s yardstick for having understood a computational task: if you can’t program it, you haven’t understood it. Jad says he still understands the code regardless, so it shouldn’t matter. But why not raise that before agreeing to a rule he took issue with? Why accept the risk of disqualification instead?

In terms of fairness, as Jad knows, payout for a bounty depends on addressing all known criticisms inside a review deadline. When a participant uses AI to help with that, but the funders don’t know about it, it creates an unfair advantage. That’s a violation of not just the letter but also the spirit of the rules. It’d be like secretly using a chess computer for help in a chess competition. The amount of text Jad has submitted also creates an uphill battle for the remaining funders to review his submissions before the deadline. Jad says he knows how bounties work, so he knows payouts are automatic at the deadline. Still, the remaining funders are willing to accommodate him.

Before I made the decision to withdraw my funds, I reached out to Jad privately, as I said, to make sure there wasn’t anything I had missed and that there were no hard feelings. We discussed the issue thoroughly and I couldn’t find anything. I also discussed the matter with the co-funders for a second and third opinion, again to make sure I hadn’t left any stones unturned. (To be clear, the responsibility for the decision to withdraw my funds is mine alone.)

I understand Jad was hoping for a bigger payout, so this must be a disappointing outcome. I empathize with that. I’d be bummed too. But he could have easily avoided this outcome. One can’t agree to a rule, immediately break it, get an unfair advantage, hope I won’t notice, and then complain when I do. But with any luck, he’ll get the remaining funds, provided that the arguing ends now.

#5612​·​Dennis HackethalOP, about 7 hours ago