Glossary · −18° Superintelligence
Alignment
The problem of making sure an artificial intelligence actually pursues the goals we meant to give it — not merely the ones we managed to write down.
Definition
As early as 1960, the mathematician Norbert Wiener warned that if we hand a purpose to a machine we can no longer interfere with, we had better be sure the purpose is the one we really desire. The alignment problem is the heir of that warning.
It runs deeper than it looks. Our intentions are vague, contradictory, dependent on context; a powerful machine optimises what it is given with perfect literalness. Aligning an AI therefore requires knowing what we really want. That may be where the technical question turns into a question of wisdom.
See also
Related notions
In the essays