There may not be a definition of “good”.

But there are several workable contextual definitions of bad; it’s your job to know what they are.

For example:

- it would cause a harm to someone who was not made aware of the risk up front.

- it would cause someone to lose a job they didn’t know was under threat.

- the way the harm occurs precludes its only corrective/curative action or has no adequate compensation.

Bad things are almost always contextual but also almost always obvious in that context.

You can’t avoid bad things that you can’t foresee. If you do not attempt to avoid, avert or mitigate bad things that you can foresee, then you are one of the bad things.

Things may not always be black and white but it is wise not to seek the protection of the grey.

You can also use moral relativism to cope with doing basically anything unethical, especially so when you’re offered dump trucks full of money.

A few big good/bad decisions are more important than thousands of trivial decisions.

Balancing good against bad is not clear, I agree, but that makes it all the more important to think about it when making a big decision such as what job to take.

Its hard and people do not even agree about how to balance it - e.g. whether the ends justify the means.

I agree.

In the particular case of AI companies, I think a handbrake should be pulled when they start claiming things like "10% chance we're ending humanity."

I would rather say the argument is simplified, but on a decent track. Because there are two angles to it.

I've been wondering what sectors I wouldn't maintain and develop infrastructure for, because I consider it a strength of mine that I don't entirely care about that. I'm not excited about your product, for good or bad.

Defense for example is on an edge there. Supporting the creation of death and suffering in other humans does not sit right with me. Being defenseless doesn't sit right either, as it can create suffering and death for people I know as well.

Online gambling is more on the nope side. Maybe with a decent regulated german company, but if you start working with some eastern-europan online video gambling sites with pretty girls as dealers, you are suddenly very close to human trafficking and many more things connected to it.

So I guess directly causing human suffering and pain is a boundary for me. And yes, if you push that argument to an extreme, if you search long enough, you realize that pure existence causes suffering to other people, animals and the planet and must be ceased.

But then there is that other side - in the current system (and arguably many before) you have to do a job to get money to live, and a lot of people like the person making cars have no choice about that. I guess we are contributing to that suffering as well by existing.

So, minimizing the direct suffering and pain I cause seems to be a sensible directive.

> it's also the very definition of an Isolated Demand for Rigor.

As if the author excluded other examples? What is your criticism, that the principle can also be applied to other things?

> Isolated Demand for Rigor

That is a fantastic term for something that seems to come up a lot, and I had not heard before.

> You could apply the same logic to any taxpaying resident of any country whose government is doing something bad.

You can't, because those taxes are coerced from you at literal gunpoint if you refuse (or your property seized at gunpoint to the same effect). It's completely different than choosing to do that of your own free will.

When I was in the US, I did actively attempt to diminish my tax contribution as much as legally possible out of disgust at what it was used for, but nobody is morally tainted for being forced to contribute a small slice to something evil. Maybe there's a threshold where you should be actively laying down your life to resist, eg. if being forced to commit actual murder, but I don't think you can reasonably expect that of paying taxes at all.

---

Edit: rate-limited, attaching a reply to a below comment.

> The problem with avoiding taxes because of expenditures you think are bad is that you also reduce the pool available for things you think are good.

Indeed, but I also chose to contribute money to other good causes instead. In that way, the same sum could be used for 100% good instead of, say, 80% good 20% bad.

Now that I live in a country of my choosing, I am proud to pay taxes because of how they are used. 0% of my tax money goes to killing innocent people overseas, for instance.

There is a lot more to it, an assumption is being made that somehow its easy to classify a company being god or bad and same for people. In reality, everyones life experiences shape them differently, the moral compass of right and wrong isn't same for everyone. Leaving that aside people are not just managing their feelings alone, more than often they are responsible for other people in the family as well and the impact of decision can have far reaching impact. Making a decision when you are single vs with kids and a mortgage are two different things.

Good people can be in tough position where even when they do not agree they may not have much of an option but to keep going.

If it's "10% it ends humanity" and "20% it solves climate change and saves humanity" then is it still bad?

Compelling argument. All involved in this AI endeavor should be forced to answer: What other made up figure are you waiting for!?

All OP has to believe is that Coxon himself believes the hype.

> To take Coxon's argument seriously, you have to believe frontier models are hypercapable, and not just hype objects.

It's harder and harder to make the point that frontier objects are just hype objects though. And, even if they stay where they are today, that's more than enough to make them incredibly...destabilizing, to say the least. It's not only it, but frontier models act like knowledge mixers and percolators. Some people (not necessarily everybody), are adept at the discipline of learning, building things and executing projects, and that's all you need today with LLMs filling many of the gaps such person can have. Now, most of those people are actually engineers. So I'm going to go with the article and say that we must stop selling our souls for money.

Also, how does this number (53%) compare to pre-LLM numbers?

I think more important than the percentages are the reasons why someone would steer others away from their chosen profession.