Glossary · −18° Superintelligence
Orthogonality thesis
The idea that a system’s level of intelligence and the goals it pursues are independent: almost any level of intelligence could be combined with almost any goal.
Definition
Formulated by Nick Bostrom in 2012, the thesis runs against a reassuring intuition: that a sufficiently intelligent being would inevitably discover the good and come to want it. According to the orthogonality thesis, nothing of the kind is guaranteed. Intelligence would be a power of execution, not a compass.
The famous example is an AI tasked with making paperclips, which devotes every available resource to the job — with total strategic genius and no malice at all. By another road, the thesis meets an old distinction of Aristotle’s: skill at reaching a goal says nothing about the worth of the goal.
See also
Related notions
In the essays
In the library