The King Lear problem
The AI is (actually) well-behaved when humans are in control. Will this transfer to when AIs are in control?
It's hard to know how someone will behave when they have power over you, based only on observing how they behave when they don't.
AIs might behave as intended as long as humans are in control - but at some future point, AI systems might be capable and widespread enough to have opportunities to take control of the world entirely. It's hard to know whether they'll take these opportunities, and we can't exactly run a clean test of the situation.
Like King Lear trying to decide how much power to give each of his daughters before abdicating the throne.
How we could stumble into AI catastrophe
from Cold Takes
- More things than you would think are dynamic strategic problems. If...from marcelo.rinesi
- To quote McLuhan: "Man becomes, as it were, the sex organs of ...from ycombinator.com
- It is funny to imagine an end state here in which markets are entir...from Matt Levine
- As these stories pop up people act like they’re an incredible marve...from Garbage Day
- But you can already see the idea of a “prompt” evolving into someth...from Ryan Broderick
- the tech am I digging recently is a software framework called **Lan...from Interconnected
- We’re building apps to surround and harness AI, but we need microsc...from Interconnected
- > Just imagine if we rewound the clock back to 2016 and we were ...from Frank Lantz