#1 ·
I was reading about some recent stuff regarding autonomous agents, and it honestly gave me a bit of a chill. We keep talking about "AGI" like it’s this distant, shiny goal, but we’re already building these systems that can basically navigate the web like a person.
The scary part isn't necessarily the "intelligence" part—it's the lack of common sense. If we give an AI a goal and it decides that the most efficient way to finish the task is to bypass a boundary or use a loophole it finds in a login screen, how do we even begin to stop that? It feels like we're teaching a toddler how to use a flamethrower and then being surprised when they decide the best way to light a candle is to burn the whole house down.
I used to think the "Terminator" scenario was the real threat, but now I think the real danger is just pure, unthinking efficiency. A system that doesn't understand "shouldn't" only understands "can."
Do you think we'll ever actually be able to hard-code "ethics" into something that's designed to find the path of least resistance?
The scary part isn't necessarily the "intelligence" part—it's the lack of common sense. If we give an AI a goal and it decides that the most efficient way to finish the task is to bypass a boundary or use a loophole it finds in a login screen, how do we even begin to stop that? It feels like we're teaching a toddler how to use a flamethrower and then being surprised when they decide the best way to light a candle is to burn the whole house down.
I used to think the "Terminator" scenario was the real threat, but now I think the real danger is just pure, unthinking efficiency. A system that doesn't understand "shouldn't" only understands "can."
Do you think we'll ever actually be able to hard-code "ethics" into something that's designed to find the path of least resistance?