[update]
AAAaaarrrrrrrgggghhhhh!
I just read the last chapter — and it’s not finished yet! I have to wait for Yukowsky to write more chapters!
I hope he’s doing more on a regular basis. Do not start reading this unless you are a masochist.
Yudkowsky, please stop wasting your time doing stupid stuff like trying to figure out how to give AIs a sense of ethics.
Finish the damn story!
[/update]
Harry Potter and the Methods of Rationality, by Eliezer Yudkowsky.
Via Eric S. Raymond. It’s useless quoting from the story itself, so let me just give you Raymond’s review:
Oh Thoth Trismegistus, oh Ma’at, oh Ganesha, oh sweet lady Eris…I have not laughed so hard in years!
Eliezer Yudkowsky is one of the brightest people I’ve ever met in a lifetime of seeking out gifted- to genius-grade thinkers because people who aren’t usually bore me pretty quickly. Eliezer has spent years studying the deep structure of rationality and probably understands the systematic sources of bias and irrationality in the shared architecture of the human mind as comprehensively as anyone alive. I have previously commented on some of his writings.
Usually Eliezer thinks about questions like how to build human-compatible ethical reasoning into AIs. Serious, deep stuff. When he turns the vast and imponderable force of his intellect to writing, of all things, Harry Potter fanfic, a quite unexpected degree of hilarity ensues.
Read it and laugh. Read it and learn. Eliezer re-invents Harry Potter as a skeptic genius who sets himself the task of figuring out just how all this “magic” stuff works. The science is real – it really would be a lot harder to explain transformation from a human into a cat than mere levitation, for example. When Harry, confronted with a magical time-travel device, is immediately terrified that he might be holding an antimatter bomb, this is actually a more justified fear than many readers may understand.
But the characters are not slighted. Eliezer is very good at giving them responses to the rather altered and powered-up Harry that are consistent with canon. The development of Minerva McGonagall is particularly fine.
Strongly recommended. And if you manage to learn about sources of cognitive bias like the Planning Fallacy and the Bystander Effect (among others) while your sides are hurting with laughter, so much the better.
It helps if you have some familiarity with the Potter cycle, but since that is itself a mish-mash of traditional child’s fantasy, you probably will recognize most what’s being built on here.
And it’s what’s being built that you need to read anyway. Gods, I wish I’d this when I was twelve.
No, wait: one quote from the story:
…It is a sad rule that whenever you are most in need of your art as a rationalist, that is when you are most likely to forget it.