Torture Data Quotes

If you torture the data long enough, it will confess.

data mining means “torturing the data until it confesses.

Torture the data, and it will confess to anything,” as

During the dot-com bubble, most people did not use a persuasive theory to gauge whether stock prices were too high, too low, or just right. Instead, as they watched stock prices go up, they invented explanations to rationalize what was happening. They talked about Moore’s Law, smart kids, and Alan Greenspan. Data without theory.

if you torture the data long enough, it will confess to anything

torturing the data until it confesses,

There are many people with an agenda out there, and they will ruthlessly torture statistics until they get the answer they want out of them.

There is always so much talk and data on the Holocaust but very little if anything is known about the blacks who were also there and tortured and experimented on.” “Yes! The Rhineland Bastards, they were called.

I mean, take for instance all this civil liberties crap. You know what I’d do if I were in power again? I’d say, okay then, we’ll have two queues at the airports. On the left, we’ll have queues to flights on which we’ve done no background checks on the passengers, no profiling, no biometric data, nothing that infringed anyone’s precious civil liberties, used no intelligence obtained under torture—nothing. On the right, we’ll have queues to the flights where we’ve done everything possible to make them safe for passengers. Then people can make their own minds up which plane they want to catch. Wouldn’t that be great? To sit back and watch which queue the Rycarts of this world would really choose to put their kids on, if the chips were down?

Our inherited desire to explain what we see fuels two kinds of cognitive errors. First, we are too easily seduced by patterns and by the theories that explain them. Second, we latch onto data that support our theories and discount contradicting evidence. We believe stories simply because they are consistent with the patterns we observe and, once we have a story, we are reluctant to let it go.

Another simile is that of the man who was born and raised in a prison and who has never set foot outside. All he knows is prison life. He would have no conception of the freedom that is beyond his world. And he would not understand that prison is suffering. If anybody suggested that his world was dukkha, he would disagree, for prison is the limit of his experience. But one day he might find the escape tunnel dug long ago that leads beyond the prison walls to the unimaginable and expansive world of real freedom. Only when he has entered that tunnel and escaped from his prison does he realize how much suffering prison actually was, and the end of that suffering, escaping from jail is happiness. In this simile the prison is the body, the high prison walls are the five senses, and the relentless demanding prison guard is one's own will, the doer. The tunnel dug long ago, through which one escapes, is called jhana [meditation] (as at AN IX, 42). Only when one has experienced jhana does one realize that the five-sense world, even at its best, is really a five-walled prison, some parts of it is a little more comfortable but still a jail with everyone on death row! Only after deep jhana does one realize that "will" was the torturer, masquerading as freedom, but preventing one ever resting happily at peace. Only outside of prison can one gain the data that produces the deep insight that discovers the truth about dukkha. In summary, without experience of jhana, one's knowledge of the world is too limited to fully understand dukkha, as required by the first noble truth, and proceed to enlightenmen.

There is a virtual consensus among scholars who study Jesus' resurrection that, subsequent to Jesus' death by crucifixion, his disciples really believed that he appeared to them risen from the dead. This conclusion has been reached by data that suggest that (1) the disciples themselves claimed that the risen Jesus had appeared to them, and (2) subsequent to Jesus' death by crucifixion, his disciples were radically transformed from fearful, cowering individuals who denied and abandoned him at his arrest and execution into bold proclaimers of the gospel of the risen Lord. They remained steadfast in the face of imprisonment, torture, and martyrdom. It is very clear that they sincerely believed that Jesus rose from the dead.

If the past is a foreign country, it is a shockingly violent one. It is easy to forget how dangerous life used to be, how deeply brutality was once woven into the fabric of daily existence. Cultural memory pacifies the past, leaving us with pale souvenirs whose bloody origins have been bleached away. A woman donning a cross seldom reflects that this instrument of torture was a common punishment in the ancient world; nor does a person who speaks of a whipping boy ponder the old practice of flogging an innocent child in place of a misbehaving prince. We are surrounded by signs of the depravity of our ancestors’ way of life, but we are barely aware of them. Just as travel broadens the mind, a literal-minded tour of our cultural heritage can awaken us to how differently they did things in the past. In a century that began with 9/11, Iraq, and Darfur, the claim that we are living in an unusually peaceful time may strike you as somewhere between hallucinatory and obscene. I know from conversations and survey data that most people refuse to believe it.1 In succeeding chapters I will make the case with dates and data. But first I want to soften you up by reminding you of incriminating facts about the past that you have known all along. This is not just an exercise in persuasion. Scientists often probe their conclusions with a sanity check, a sampling of real-world phenomena to reassure themselves they haven’t overlooked some flaw in their methods and wandered into a preposterous conclusion. The vignettes in this chapter are a sanity check on the data to come.

In reality, evangelicals did not cast their vote despite their beliefs, but because of them. Donald Trump did not trigger this militant turn; his rise was symptomatic of a long-standing condition. Survey data reveal the stark contours of the contemporary evangelical worldview. More than any other religious demographic in America, white evangelical Protestants support preemptive war, condone the use of torture, and favor the death penalty. They are more likely than members of other faith groups to own a gun, to believe citizens should be allowed to carry guns in most places, and to feel safer with a firearm around. White evangelicals are more opposed to immigration reform and have more negative views of immigrants than any other religious demographic; two-thirds support Trump’s border wall. Sixty-eight percent of white evangelical Protestants—more than any other demographic—do not think that the United States has a responsibility to accept refugees. More than half of white evangelical Protestants think a majority nonwhite US population would be a negative development. White evangelicals are considerably more likely than others to believe that Islam encourages violence, to refuse to see Islam as “part of mainstream American society,” and to perceive “natural conflict between Islam and democracy.” At the same time, white evangelicals believe that Christians in America face more discrimination than Muslims. White evangelicals are significantly more authoritarian than other religious groups, and they express confidence in their religious leaders at much higher rates than do members of other faiths.

In September 1999, the Department of Justice succeeded in denaturalizing 63 participants in Nazi acts of persecution; and in removing 52 such individuals from this country. This appears to be but a small portion of those who actually were brought here by our own government. A 1999 report to the Senate and the House said "that between 1945 and 1955, 765 scientists, engineers, and technicians were brought to the United States under Overcast, Paperclip, and similar programs. It has been estimated that at least half, and perhaps as many as 80 percent of all the imported specialists were former Nazi Party members." A number of these scientists were recruited to work for the Air Force's School of Aviation Medicine (SAM) at Brooks Air Force Base in Texas, where dozens of human radiation experiments were conducted during the Cold War. Among them were flash-blindness studies in connection with atomic weapons tests and data gathering for total-body irradiation studies conducted in Houston. The experiments for which Nazi investigators were tried included many related to aviation research. Hubertus Strughold, called "the father of space medicine," had a long career at the SAM, including the recruitment of other Paperclip scientists in Germany. On September 24, 1995 the Jewish Telegraphic Agency reported that as head of Nazi Germany's Air Force Institute for Aviation Medicine, Strughold particpated in a 1942 conference that discussed "experiments" on human beings. The experiments included subjecting Dachau concentration camp inmates to torture and death. The Edgewood Arsenal of the Army's Chemical Corps as well as other military research sites recruited these scientists with backgrounds in aeromedicine, radiobiology, and opthamology. Edgewood Arsenal, Maryland ended up conducting experiments on more than seven thousand American soldiers. Using Auschwitz experiments as a guide, they conducted the same type of poison gas experiments that had been done in the secret I.G. Farben laboratories.

Henry, there’s something I would like to tell you, for what it’s worth, something I wish I had been told years ago. You’ve been a consultant for a long time, and you’ve dealt a great deal with top secret information. But you’re about to receive a whole slew of special clearances, maybe fifteen or twenty of them, that are higher than top secret. I’ve had a number of these myself, and I’ve known other people who have just acquired them, and I have a pretty good sense of what the effects of receiving these clearances are on a person who didn’t previously know they even existed. And the effects of reading the information that they will make available to you. First, you’ll be exhilarated by some of this new information, and by having it all—so much! incredible!—suddenly available to you. But second, almost as fast, you will feel like a fool for having studied, written, talked about these subjects, criticized and analyzed decisions made by presidents for years without having known of the existence of all this information, which presidents and others had and you didn’t, and which must have influenced their decisions in ways you couldn’t even guess. In particular, you’ll feel foolish for having literally rubbed shoulders for over a decade with some officials and consultants who did have access to all this information you didn’t know about and didn’t know they had, and you’ll be stunned that they kept that secret from you so well. You will feel like a fool, and that will last for about two weeks. Then, after you’ve started reading all this daily intelligence input and become used to using what amounts to whole libraries of hidden information, which is much more closely held than mere top secret data, you will forget there ever was a time when you didn’t have it, and you’ll be aware only of the fact that you have it now and most others don’t … and that all those other people are fools. Over a longer period of time—not too long, but a matter of two or three years—you’ll eventually become aware of the limitations of this information. There is a great deal that it doesn’t tell you, it’s often inaccurate, and it can lead you astray just as much as the New York Times can. But that takes a while to learn. In the meantime it will have become very hard for you to learn from anybody who doesn’t have these clearances. Because you’ll be thinking as you listen to them: “What would this man be telling me if he knew what I know? Would he be giving me the same advice, or would it totally change his predictions and recommendations?” And that mental exercise is so torturous that after a while you give it up and just stop listening. I’ve seen this with my superiors, my colleagues … and with myself. You will deal with a person who doesn’t have those clearances only from the point of view of what you want him to believe and what impression you want him to go away with, since you’ll have to lie carefully to him about what you know. In effect, you will have to manipulate him. You’ll give up trying to assess what he has to say. The danger is, you’ll become something like a moron. You’ll become incapable of learning from most people in the world, no matter how much experience they may have in their particular areas that may be much greater than yours.

In September 1999, the Department of Justice succeeded in denaturalizing 63 participants in Nazi acts of persecution; and in removing 52 such individuals from this country. This appears to be but a small portion of those who actually were brought here by our own government. A 1999 report to the Senate and the House said "that between 1945 and 1955, 765 scientists, engineers, and technicians were brought to the United States under Overcast, Paperclip, and similar programs. It has been estimated that at least half, and perhaps as many as 80 percent of all the imported specialists were former Nazi Party members." A number of these scientist were recruited to work for the Air Force's School of Aviation Medicine (SAM) at Brooks Air Force Base in Texas, where dozens of human radiation experiments were conducted during the Cold War. Among them were flash-blindness studies in connection with atomic weapons tests and data gathering for total-body irradiation studies conducted in Houston. The experiments for which Nazi investigators were tried included many related to aviation research. Hubertus Strughold, called "the father of space medicine," had a long career at the SAM, including the recruitment of other Paperclip scientists in Germany. On September 24, 1995 the Jewish Telegraphic Agency reported that as head of Nazi Germany's Air Force Institute for Aviation Medicine, Strughold particpated in a 1942 conference that discussed "experiments" on human beings. The experiments included subjecting Dachau concentration camp inmates to torture and death. The Edgewood Arsenal of the Army's Chemical Corps as well as other military research sites recruited these scientists with backgrounds in aeromedicine, radiobiology, and opthamology. Edgewood Arsenal, Maryland ended up conducting experiments on more than seven thousand American soldiers. Using Auschwitz experiments as a guide, they conducted the same type of poison gas experiments that had been done in the secret I.G. Farben laboratories.

But *why* do clean torture and democracy appear to go hand in hand? This is an important puzzle (though by no means the only one suggested by the data). My explanation for this pattern generally is this: Public monitoring leads institutions that favor painful coercion to use and combine clean torture techniques to evade detection, and, to the extent that public monitoring is not only greater in democracies, but that public monitoring of human rights is a core value in modern democracies, it is the case that where we find democracies torturing today we will also be more likely to find stealthy torture.

When a theory is generated by ransacking data, we can’t use these pillaged data to test the theory.

Finding a historical correlation between gold and silver prices is no more convincing than finding a historical correlation between marriage and beer consumption. All it demonstrates is that we looked long enough to find a correlation.

Be doubly skeptical of graphs that have two vertical axes and omit zero from either or both axes.

It’s even been said that data mining means “torturing the data until it confesses.

In carpentry, they say, “Measure twice, cut once.” With data, “Think twice, calculate once.

There are few statistical facts more interesting than regression to the mean for two reasons. First, people encounter it almost every day of their lives. Second, almost nobody understands it.

Rushing to optimize before the bottlenecks are known may be the only error to have ruined more designs than feature creep. From tortured code to incomprehensible data layouts, the results of obsessing about speed or memory or disk usage at the expense of transparency and simplicity are everywhere. They spawn innumerable bugs and cost millions of man-hours—often, just to get marginal gains in the use of some resource much less expensive than debugging time. Disturbingly often, premature local optimization actually hinders global optimization (and hence reduces overall performance). A prematurely optimized portion of a design frequently interferes with changes that would have much higher payoffs across the whole design, so you end up with both inferior performance and excessively complex code.

Kahneman later wrote: This was a joyous moment, in which I understood an important truth about the world: because we tend to reward others when they do well and punish them when they do badly, and because there is regression to the mean, it is part of the human condition that we are statistically punished for rewarding others and rewarded for punishing them.

If you torture the data enough, nature will always confess.

Ronald Coase cynically observed that, “If you torture the data long enough, it will confess.

The data always tells you the story you want to hear. “If you torture the data long enough, it will confess to anything,” according to economist Ronald Coase.

This hardened response to those on the “other team” is not an invention of modern American politics. It seems to be hardwired into the circuitry of our brains. The Old Testament is filled with stories of sometimes deadly tribalism, and scientific data gives us insight into why that happens. In 1968, elementary school teacher Jane Elliott conducted a famous experiment with her students in the days after the assassination of Dr. Martin Luther King Jr. She divided the class by eye color. The brown-eyed children were told they were better. They were the “in-group.” The blue-eyed children were told they were less than the brown-eyed children—hence becoming the “out-group.” Suddenly, former classmates who had once played happily side by side were taunting and torturing one another on the playground. Lest we assign greater morality to the “out-group,” the blue-eyed children were just as quick to attack the brown-eyed children once the roles were reversed.6

Things that cannot be described by numbers should not be described by numbers. A lot of the most important, actionable “data” in a classroom cannot be tortured into an integer, benchmark, zone, indicator, or anything else that looks good in a report.

We see the same behavior when athletes wear unwashed lucky socks, when investors buy hot stocks, or when people throw good money after bad, confident that things must take a turn for the better. We yearn to make an uncertain world more certain, to gain control over things that we do not control, to predict the unpredictable. If we did well wearing these socks, then it must be that these socks help us do well. If other people made money buying this stock, then we can make money buying this stock. If we have had bad luck, our luck has to change, right? Order is more comforting than chaos. These cognitive errors make us susceptible to all sorts of statistical deceptions. We are too quick to assume that meaningless patterns are meaningful when they are presented as evidence of the consequences of a government policy, the power of a marketing plan, the success of an investment strategy, or the benefits of a food supplement. Our vulnerability comes from a deep desire to make sense of the world, and it’s notoriously hard to shake off.

We are not interested in the authority-obeying, adapted mass. We are not interested in people who strive for a goal that aids only them. We want the dreamers, the people who think differently, who doubt themselves and others constantly. We want the ones who fail, fall, stand up, and try again. The ones who put puzzle pieces together in a way others can’t. The ones who see the large picture, who see the world and the humanity within, and not just their own small bubble of reality. We need the ones who can put anger, fear, and hate aside and analyse data independently of their own wants and needs. We want the excellent observers, the ones who are so sensitive to their surroundings, to the mass of normal people with all the ignorance they spread, that it destroys them. And of these few, we choose only the ones who never externalise their frustration. Never the ones who torture others — the weaker, the smaller, or animals in their care. We almost always choose the ones who take it out on themselves.

I once had a foreign exchange trader who worked for me who was an unabashed chartist. He truly believed that all the information you needed was reflected in the past history of a currency. Now it's true there can be less to consider in trading currencies than individual equities, since at least for developed country currencies it's typically not necessary to pore over their financial statements every quarter. And in my experience, currencies do exhibit sustainable trends more reliably than, say, bonds or commodities. Imbalances caused by, for example, interest rate differentials that favor one currency over another (by making it more profitable to invest in the higher-yielding one) can persist for years. Of course, another appeal of charting can be that it provides a convenient excuse to avoid having to analyze financial statements or other fundamental data. Technical analysts take their work seriously and apply themselves to it diligently, but it's also possible for a part-time technician to do his market analysis in ten minutes over coffee and a bagel. This can create the false illusion of being a very efficient worker. The FX trader I mentioned was quite happy to engage in an experiment whereby he did the trades recommended by our in-house market technician. Both shared the same commitment to charts as an under-appreciated path to market success, a belief clearly at odds with the in-house technician's avoidance of trading any actual positions so as to provide empirical proof of his insights with trading profits. When challenged, he invariably countered that managing trading positions would challenge his objectivity, as if holding a losing position would induce him to continue recommending it in spite of the chart's contrary insight. But then, why hold a losing position if it's not what the chart said? I always found debating such tortured logic a brief but entertaining use of time when lining up to get lunch in the trader's cafeteria. To the surprise of my FX trader if not to me, the technical analysis trading account was unprofitable. In explaining the result, my Kool-Aid drinking trader even accepted partial responsibility for at times misinterpreting the very information he was analyzing. It was along the lines of that he ought to have recognized the type of pattern that was evolving but stupidly interpreted the wrong shape. It was almost as if the results were not the result of the faulty religion but of the less than completely faithful practice of one of its adherents. So what use to a profit-oriented trading room is a fully committed chartist who can't be trusted even to follow the charts? At this stage I must confess that we had found ourselves in this position as a last-ditch effort on my part to salvage some profitability out of a trader I'd hired who had to this point been consistently losing money. His own market views expressed in the form of trading positions had been singularly unprofitable, so all that remained was to see how he did with somebody else's views. The experiment wasn't just intended to provide a “live ammunition” record of our in-house technician's market insights, it was my last best effort to prove that my recent hiring decision hadn't been a bad one. Sadly, his failure confirmed my earlier one and I had to fire him. All was not lost though, because he was able to transfer his unsuccessful experience as a proprietary trader into a new business advising clients on their hedge fund investments.