Knuth Quotes

Science is what we understand well enough to explain to a computer; art is everything else.

The computer scientist Donald Knuth was struck that “AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking’—that, somehow, is much harder!

The best programs are written so that computing machines can perform them quickly and so that human beings can understand them clearly. A programmer is ideally an essayist who works with traditional aesthetic and literary forms as well as mathematical concepts, to communicate the way that an algorithm works and to convince a reader that the results will be correct.

Premature optimization is the root of all evil.

An algorithm must be seen to be believed.

Beware of bugs in the above code; I have only proved it correct, not tried it.

Programs are meant to be read by humans and only incidentally for computers to execute.

The language in which we express our ideas has a strong influence on our thought processes.

Nearly every example of faulty reasoning that has been published is accompanied by the phrase "of course" or its equivalent.

Overstimulation has been the real drawback. I need to find ways to stop thinking about analysis of algorithms, in order to do various other things that human beings ought to do.

I have been a happy man ever since January 1, 1990, when I no longer had an email address. I’d used email since about 1975, and it seems to me that 15 years of email is plenty for one lifetime. Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration. Knuth

Premature optimization is the root of all evil.

Programming is the art of telling another human being what one wants the computer to do.

If you optimize everything, you will always be unhappy.

It's not really fun listening to an insane person. Do you realize that? You're only entertaining yourself. Not much of a host, if you don't mind my critique.

We should continually be striving to transform every art into a science: in the process, we advance the art.

Everyday life is like programming, I guess. If you love something you can put beauty into it.

By the way, if you’re wondering where the word “algorithm” comes from, Knuth says that it derives from the name “al-Khowârizmî,” a ninth-century Persian mathematician.

People think that computer science is the art of geniuses but the actual reality is the opposite, just many people doing things that build on each other, like a wall of mini stones.

وبالإضافة إلى ما تتسم به الكتب من حيوية لصيقة بجوهرها، فهي تنفخ في المجتمعات الروح؛ أما المكتبات فتنسج القصص التي تمنح حياتنا شكلًا ومعنى، فتساعد الأفراد والثقافات على تحديد وجهتهم ومعرفة ذواتهم.

Jonathon stretched his arms, like he was reading himself for exercise. He leaned back and asked, "Why are you doing this?" Robin was a bit surprised by the blunt question. "Doing what? Laying here, strapped to this table? I'll be honest, I've asked myself the same question.

Instead of concentrating just on finding good answers to questions, it's more important to learn how to find good questions!

In fact, my experiences as I was writing the 3:16 book weren't that different from writing computer books, although I wasn't using integral signs as much.

Far from being a mere product of evil, the destruction of textual materials was goal-oriented and carefully rationalized within struggles between competing worldviews that wracked the last century.

Deep within the web of pavement and steel, a young boy and girl scurried past an old man, his outdated mechanical limbs shaking and twitching like an addict without a fix. He scowled at the hoodlums, scratching the hole in his face where his nose used to be, sold long ago as a cheap replacement part for someone slightly richer than he was.

A person’s success in life is determined by having a high minimum, not a high maximum. If you can do something really well but there are other things at which you’re failing, the latter will hold you back. But if almost everything you do is up there, then you’ve got a good life. And so I try to learn how to get through things that others find unpleasant.

Knuth: They were very weak, actually. It wasn't presented systematically and everything, but I thought they were pretty obvious. It was a different culture entirely. But the guy who said he was going to fire people, he wants programming to be something where everything is done in an inefficient way because it's supposed to fit into his idea of orderliness. He doesn't care if the program is good or not—as far as its speed and performance—he cares about that it satisfies other criteria, like any bloke can be able to maintain it. Well, people have lots of other funny ideas. People have this strange idea that we want to write our programs as worlds unto themselves so that everybody else can just set up a few parameters and our program will do it for them. So there'll be a few programmers in the world who write the libraries, and then there are people who write the user manuals for these libraries, and then there are people who apply these libraries and that's it. The problem is that coding isn't fun if all you can do is call things out of a library, if you can't write the library yourself. If the job of coding is just to be finding the right combination of parameters, that does fairly obvious things, then who'd want to go into that as a career? There's this overemphasis on reusable software where you never get to open up the box and see what's inside the box. It's nice to have these black boxes but, almost always, if you can look inside the box you can improve it and make it work better once you know what's inside the box. Instead people make these closed wrappers around everything and present the closure to the programmers of the world, and the programmers of the world aren't allowed to diddle with that. All they're able to do is assemble the parts. And so you remember that when you call this subroutine you put x0, y0, x1, y1 but when you call this subroutine it's x0, x1, y0, y1. You get that right, and that's your job.

Nearly every example of faulty reasoning that has been published is accompanied by the phrase "of course" or its equivalent.

The language in which we express our ideas has a strong influence on our thought processes.

An algorithm must be seen to be believed

The five stages of model development. —Donald Knuth, Stanford computer scientist Knuth discovered that computer program development goes through five stages. These steps also apply to building models, and I rigorously adhere to them in my consulting work. 1. Decide what you want the model to do. 2. Decide how to build the model. 3. Build the model. 4. Debug the model. 5. Trash stages 1 through 4 and start again, now that you know what you really wanted in the first place. Once you realize that step 5 is inevitable, you become more willing to discard bad models early rather than continually to patch them up. In fact, I recommend getting to step 5 many times by building an evolving set of prototypes. This is consistent with an emerging style of system development known as Extreme Programming.2 To get a large model to work you must start with a small model that works, not a large model that doesn’t work. —Alan Manne, Stanford energy economist

premature optimization is the root of all evil. —Donald E. Knuth [

From now on whenever I read a math book, I'm going to try to figure out by myself how everything was done, before looking at the solution. Even if I don't figure it out, I think I'll be able to see the beauty of a proof then.

The most important thing in the programming language is the name. A language will not succeed without a good name. I have recently invented a very good name and now I am looking for a suitable language.

The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.

In fact the author of The Art of Computer Programming, the field’s bible (and some call Knuth himself “the Yoda of Silicon Valley”), recently commented that he feels “algorithms are getting too prominent in the world. It started out that computer scientists were worried nobody was listening to us. Now I’m worried that too many people are listening.

Seibel: Are there any books that you think all programmers should read? Norvig: I think there are a lot of choices. I don't think there's only one path. You've got to read some algorithm book. You can't just pick these things out and paste them together. It could be Knuth, or it could be the Cormen, Leiserson, and Rivest. And there are others. Sally Goldman's here now. She has a new book out that's a more practical take on algorithms. I think that's pretty interesting. So you need one of those. You need something on the ideas of abstraction. I like Abelson and Sussman. There are others.

Now, why hasn't this spread over the whole world and why isn't everybody doing it? I'm not sure who it was who hit the nail on the head—I think it was Jon Bentley. Simplified it is like this: only two percent of the world's population is born to be super programmers. And only two percent of the population is born to be super writers. And Knuth is expecting everybody to be both. I don't think we're going to increase the total number of programmers in the world to more than two percent—I mean programmers who really resonate with the machine and that's their bread and butter that they've been born to do. But now that people are blogging, I've seen a great rise in the average ability of ordinary everybody to express themselves. So the second part of that that argument isn't so strong anymore.

Seibel: In 1974 you said that by 1984 we would have “Utopia 84,” the sort of perfect programming language, and it would supplant COBOL and Fortran, and you said then that there were indications that such language is very slowly taking shape. It's now a couple of decades since '84 and it doesn't seem like that's happened. Knuth: No. Seibel: Was that just youthful optimism? Knuth: I was thinking about Simula and trends in object-oriented programming when I wrote that, clearly. I think what happens is that every time a new language comes out it cleans up what's understood about the old languages and then adds something new, experimental and so on, and nobody has ever come to the point where they have a new language and then they want to stop at what's understood. They're always wanting to push further. Maybe someday somebody will say, “No, I'm not going to be innovative; I'm just going to be clean and simple, and I'm going to stick to it.” Pascal was started with that philosophy but then didn't continue. Maybe we'll get to a time when somebody will say, “Let's set our sights lower and really try to make something that's going to be stable.” It might be a good idea.

Steele: So I guess there's lessons there—the lesson I should have drawn is there may be more than one bug here and I should have looked harder the first time. But another lesson is that if a bug is thought to be rare, then looking at rarely executed paths may be fruitful. And a third thing is, having good documentation about what the algorithm is trying to do, namely a reference back to Knuth, was just great.

Thinking about small things every day from today will lead you to rise to the ranks of Donald Knuth and Alan Turing in a few years.

Seibel: What's your desert-island list of books for programmers? Peyton Jones: Well, you should definitely read Jon Bentley's Programming Pearls. Speaking of pearls, Brian Hayes has a lovely chapter in this book Beautiful Code entitled, “Writing Programs for ‘The Book’” where I think by “The Book” he means a program that will have eternal beauty. You've got two points and a third point and you have to find which side of the line between the two points this third point is on. And several solutions don't work very well. But then there's a very simple solution that just does it right. Of course, Don Knuth's series, The Art of Computer Programming. I don't think it was ever anything I read straight through; it's not that kind of book. I certainly referred to it a lot at one stage. Chris Okasaki's book Purely Functional Data Structures. Fantastic. It's like Arthur Norman's course only spread out to a whole book. It's about how you can do queues and lookup tables and heaps without any side effects but with good complexity bounds. Really, really nice book. Everyone should read this. It's also quite short and accessible as well. Structure and Interpretation of Computer Programs. Abelson and Sussman. I loved that. And Compiling with Continuations, Andrew Appel's book about how to compile a functional program using continuation passing style. Also wonderful. Books that were important to me but I haven't read for a long time: A Discipline of Programming by Dijkstra. Dijkstra is very careful about writing beautiful programs. These ones are completely imperative but they have the “Hoare property” of rather than having no obvious bugs they obviously have no bugs. And it gives very nice, elegant reasoning to reason about it. That's a book that introduced me for the first time to reasoning about programs in a pretty watertight way. Another book that at the time made a huge impression on me was Per Brinch Hansen's book about writing concurrent operating systems. I read it lots of times.

AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking.

you visit Knuth’s website at Stanford with the intention of finding his e-mail address, you’ll instead discover the following note: I have been a happy man ever since January 1, 1990, when I no longer had an email address. I’d used email since about 1975, and it seems to me that 15 years of email is plenty for one lifetime. Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration.

Knuth goes on to acknowledge that he doesn’t intend to cut himself off completely from the world. He notes that writing his books requires communication with thousands of people and that he wants to be responsive to questions and comments. His solution? He provides an address—a postal mailing address. He says that his administrative assistant will sort through any letters arriving at that address and put aside those that she thinks are relevant. Anything that’s truly urgent she’ll bring to Knuth promptly, and everything else he’ll handle in a big batch, once every three months or so.

People who discover the power and beauty of high-level, abstract ideas often make the mistake of believing that concrete ideas at lower levels are worthless and might as well be forgotten. On the contrary, the best computer scientists are thoroughly grounded in basic concepts of how computers actually work. The essence of computer science is an ability to understand many levels of abstraction simultaneously.

Knuth has not had an email address since 1990. “Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration.” He reviews all his postal mail every three months, and all his faxes every six.

the lesson I should have drawn is there may be more than one bug here and I should have looked harder the first time. But another lesson is that if a bug is thought to be rare, then looking at rarely executed paths may be fruitful. And a third thing is, having good documentation about what the algorithm is trying to do, namely a reference back to Knuth, was just great.

I don’t know that you’re that far out on the spectrum, at least among the people I’ve talked to for this book. Though Don Knuth did write TeX in pencil in a notebook for six months before he typed in a line of code and he said he saved time because he didn’t have to bother writing scaffolding to test all the code he was developing because he just wrote the whole thing.

I'm trying to get to the bottom of things

I'm trying to get to the bottom of things.

Everyday life is like programming, I guess.If you love something you can put beauty on it.

Everyday life is like programming, I guess. If you love something you can put beauty on it.

If you find that you're spending almost all your time on theory, start turning some attention to practical things; it will improve your theories. If you find that you're spending almost all your time on practice, start turning some attention to theoretical things; it will improve your practice.

Knuth deploys what I call the monastic philosophy of deep work scheduling. This philosophy attempts to maximize deep efforts by eliminating or radically minimizing shallow obligations. Practitioners of the monastic philosophy tend to have a well-defined and highly valued professional goal that they’re pursuing, and the bulk of their professional success comes from doing this one thing exceptionally well.

In the beginning, everything was void, and J. H. W. H. Conway began to create numbers. Conway said, "Let there be two rules which bring forth all numbers large and small. This shall be the first rule: Every number corresponds to two sets of previously created numbers, such that no member of the left set is greater than or equal to any member of the right set. And the second rule shall be this: One number is less than or equal to another number if and only if no member of the first number's left set is greater than or equal to the second number, and no member of the second number's right set is less than or equal to the first number." And Conway examined these two rules he had made, and behold! They were very good. … And Conway said, "Let the numbers be added to each other in this wise: The left set of the sum of two numbers shall be the sums of all left parts of each number with the other; and in like manner the right set shall be from the right parts, each according to its kind." Conway proved that every number plus zero is unchanged, and he saw that addition was good. And the evening and the morning were the third day. And Conway said, "Let the negative of a number have as its sets the negatives of the number's opposite sets; and let subtraction be addition of the negative." And it was so. Conway proved that subtraction was the inverse of addition, and this was very good. And the evening and the morning were the fourth day. And Conway said to the numbers, "Be fruitful and multiply. Let part of one number be multiplied by another and added to the product of the first number by part of the other, and let the product of the parts be subtracted. This shall be done in all possible ways, yielding a number in the left set of the product when the parts are of the same kind, but in the right set when they are of opposite kinds." Conway proved that every number times one is unchanged. And the evening and the morning were the fifth day. And behold! When the numbers had been created for infinitely many days, the universe itself appeared. And the evening and the morning were N day. And Conway looked over all the rules he had made for numbers, and saw that they were very, very good.

I have only proven the algorithm correct, not tested it.

It’s very important to be able to get inside of somebody else’s way of thinking, to decode their vocabulary, their notation. If you can understand something about the way they thought and the way they made a discovery, then that helps you make your own discoveries.

A brute-force solution that works is better than an elegant solution that doesn't work. It can take a long time to get an elegant solution to work. In describing the history of searching algorithms, for example, Donald Knuth pointed out that even though the first description of a binary search algorithm was published in 1946, it took another 16 years for someone to publish an algorithm that correctly searched lists of all sizes (Knuth 1998). A binary search is more elegant, but a brute-force, sequential search is often sufficient. When in doubt, use brute force. — Butler Lampson

For software developers who take their work seriously, Knuth is the consummate craftsman. Here’s what he has to say about optimization: Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.2 (Emphasis added.)

The PLT helped us to understand what our programming instructions were doing along the essential axis of speed and showed us precisely when and where we were introducing slowness to our source code. The PLT told us when to pay attention to the “small efficiencies” Knuth mentioned. It was our 3 percent escape hatch, a way to know for sure that optimization wasn’t “premature.” We were sure each optimization we did was helping to keep performance heading in the right direction.

So I went to Case, and the Dean of Case says to us, says, it’s a all men’s school, says, “Men, look at, look to the person on your left, and the person on your right. One of you isn’t going to be here next year; one of you is going to fail.” So I get to Case, and again I’m studying all the time, working really hard on my classes, and so for that I had to be kind of a machine. I, the calculus book that I had, in high school we — in high school, as I said, our math program wasn’t much, and I had never heard of calculus until I got to college. But the calculus book that we had was great, and in the back of the book there were supplementary problems that weren’t, you know, that weren’t assigned by the teacher. The teacher would assign, so this was a famous calculus text by a man named George Thomas, and I mention it especially because it was one of the first books published by Addison-Wesley, and I loved this calculus book so much that later I chose Addison-Wesley to be the publisher of my own book. But Thomas’s Calculus would have the text, then would have problems, and our teacher would assign, say, the even numbered problems, or something like that. I would also do the odd numbered problems. In the back of Thomas’s book he had supplementary problems, the teacher didn’t assign the supplementary problems; I worked the supplementary problems. I was, you know, I was scared I wouldn’t learn calculus, so I worked hard on it, and it turned out that of course it took me longer to solve all these problems than the kids who were only working on what was assigned, at first. But after a year, I could do all of those problems in the same time as my classmates were doing the assigned problems, and after that I could just coast in mathematics, because I’d learned how to solve problems. So it was good that I was scared, in a way that I, you know, that made me start strong, and then I could coast afterwards, rather than always climbing and being on a lower part of the learning curve.

The process of preparing programs for a digital computer is especially attractive, not only because it can be economically and scientifically rewarding, but also because it can be an aesthetic experience much like composing poetry or music.

B: True. I've got this made urge to get up before a class and present our results: Theorem, proof, lemma, remark. I'd make it so slick nobody would be able to guess how we did it, and everyone would be

B: True. I've got this mad urge to get up before a class and present our results: Theorem, proof, lemma, remark. I'd make it so slick nobody would be able to guess how we did it, and everyone would be so impressed.