On the topic of alignment, I think you’re thinking of alignment with human values, which I think you’re right is impossible. For that matter, humans aren’t aligned with human values. But you might be able to make an AI aligned to a well defined goal, in the same sort of way your car is aligned to moving forwards when you press the gas pedal, assuming it isn’t broken. Then it becomes a matter of us quibbling about what that goal should be. Also making it well defined, which we currently suck at. As a simple example, imagine we use an AGI to build a video game. I don’t see a fundamental reason we couldn’t align AGIs to building good video games that people enjoy. Granted even in that case I’m not convinced alignment is possible, I’m just arguing that it might be.
On the topic of life as the goal, I agree. Life by default involves a lot of suffering, which is not great. I also think there’s a question of whether sentient and/or intelligent life is more valuable than non-sentient life.
On the topic of alignment, I think you’re thinking of alignment with human values, which I think you’re right is impossible. For that matter, humans aren’t aligned with human values. But you might be able to make an AI aligned to a well defined goal, in the same sort of way your car is aligned to moving forwards when you press the gas pedal, assuming it isn’t broken. Then it becomes a matter of us quibbling about what that goal should be. Also making it well defined, which we currently suck at. As a simple example, imagine we use an AGI to build a video game. I don’t see a fundamental reason we couldn’t align AGIs to building good video games that people enjoy. Granted even in that case I’m not convinced alignment is possible, I’m just arguing that it might be.
On the topic of life as the goal, I agree. Life by default involves a lot of suffering, which is not great. I also think there’s a question of whether sentient and/or intelligent life is more valuable than non-sentient life.