Reflections on Trusting Trust (Turing Award Lecture)
Enjoyment
8
Importance
--
Date Added
1.6.26
TLDR
Any system used to build/verify/interpret itself (e.g., a self-hosting compiler) can be used to silently add a hidden behavior that persists even when future source codes are clean. More generally, it is more important to trust the people who wrote the software than the source code, because behaviors can always be hidden in lower levels.
2 Cents
It took me a while to understand but I loved the talk from the designer of Unix. He frames it nicely by showing (1) programs can be self-reproducing, (2) a self-referencing compiler can make a behavior persist across recompilations, and (3) that same self-reference can be abused to decouple source code from a program’s true behavior.
Tags
Technically, Stage 1 (self-reproducing programs) is not the reason Stage 2 (self-hosting compilers) works (it's just a nice presentation of ideas). Self-hosting compilers exploit a weaker form of self-reference: a program can process descriptions of programs written in its own language.
Stage 3 requires ideas from both Stage 1 and Stage 2, because to persist itself the compiler needs to reproduce itself in any compilers it compiles.