Software Testing Quotes

The test of the machine is the satisfaction it gives you. There isn't any other test. If the machine produces tranquility it's right. If it disturbs you it's wrong until either the machine or your mind is changed.

I’m a modern man, a man for the millennium. Digital and smoke free. A diversified multi-cultural, post-modern deconstruction that is anatomically and ecologically incorrect. I’ve been up linked and downloaded, I’ve been inputted and outsourced, I know the upside of downsizing, I know the downside of upgrading. I’m a high-tech low-life. A cutting edge, state-of-the-art bi-coastal multi-tasker and I can give you a gigabyte in a nanosecond! I’m new wave, but I’m old school and my inner child is outward bound. I’m a hot-wired, heat seeking, warm-hearted cool customer, voice activated and bio-degradable. I interface with my database, my database is in cyberspace, so I’m interactive, I’m hyperactive and from time to time I’m radioactive. Behind the eight ball, ahead of the curve, ridin the wave, dodgin the bullet and pushin the envelope. I’m on-point, on-task, on-message and off drugs. I’ve got no need for coke and speed. I've got no urge to binge and purge. I’m in-the-moment, on-the-edge, over-the-top and under-the-radar. A high-concept, low-profile, medium-range ballistic missionary. A street-wise smart bomb. A top-gun bottom feeder. I wear power ties, I tell power lies, I take power naps and run victory laps. I’m a totally ongoing big-foot, slam-dunk, rainmaker with a pro-active outreach. A raging workaholic. A working rageaholic. Out of rehab and in denial! I’ve got a personal trainer, a personal shopper, a personal assistant and a personal agenda. You can’t shut me up. You can’t dumb me down because I’m tireless and I’m wireless, I’m an alpha male on beta-blockers. I’m a non-believer and an over-achiever, laid-back but fashion-forward. Up-front, down-home, low-rent, high-maintenance. Super-sized, long-lasting, high-definition, fast-acting, oven-ready and built-to-last! I’m a hands-on, foot-loose, knee-jerk head case pretty maturely post-traumatic and I’ve got a love-child that sends me hate mail. But, I’m feeling, I’m caring, I’m healing, I’m sharing-- a supportive, bonding, nurturing primary care-giver. My output is down, but my income is up. I took a short position on the long bond and my revenue stream has its own cash-flow. I read junk mail, I eat junk food, I buy junk bonds and I watch trash sports! I’m gender specific, capital intensive, user-friendly and lactose intolerant. I like rough sex. I like tough love. I use the “F” word in my emails and the software on my hard-drive is hardcore--no soft porn. I bought a microwave at a mini-mall; I bought a mini-van at a mega-store. I eat fast-food in the slow lane. I’m toll-free, bite-sized, ready-to-wear and I come in all sizes. A fully-equipped, factory-authorized, hospital-tested, clinically-proven, scientifically- formulated medical miracle. I’ve been pre-wash, pre-cooked, pre-heated, pre-screened, pre-approved, pre-packaged, post-dated, freeze-dried, double-wrapped, vacuum-packed and, I have an unlimited broadband capacity. I’m a rude dude, but I’m the real deal. Lean and mean! Cocked, locked and ready-to-rock. Rough, tough and hard to bluff. I take it slow, I go with the flow, I ride with the tide. I’ve got glide in my stride. Drivin and movin, sailin and spinin, jiving and groovin, wailin and winnin. I don’t snooze, so I don’t lose. I keep the pedal to the metal and the rubber on the road. I party hearty and lunch time is crunch time. I’m hangin in, there ain’t no doubt and I’m hangin tough, over and out!

Software testing is a sport like hunting, it's bughunting.

If it hurts, do it more frequently, and bring the pain forward.

All software you write will be tested—if not by you and your team, then by the eventual users—so you might as well plan on testing it thoroughly.

There are better versions of me, Jeremy. It's not like with people. With people you can argue and have tests and music reviews and wars to decide who's better, but with software, it's pretty clear. I get evolved beyond my version number, and then I'm useless.

At Apple, we never would have dreamed of doing that, and we never staged any A/ B tests for any of the software on the iPhone. When it came to choosing a color, we picked one. We used our good taste—and our knowledge of how to make software accessible to people with visual difficulties related to color perception—and we moved on.

If you don't have questions about a product's risks, then there's no reason to test. If you have at least one such question, then ask: Will these tests cost more to execute than their answers will be worth?

Testing gathers information about a product; it does not fix things it finds that are wrong.

RICH PURNELL sipped coffee in the silent building. He ran a final test on the software he’d written. It passed. With a relieved sigh, he sank back in his chair. Checking the clock on his computer, he shook his head. 3:42 a.m.

Zues?" I said. "His computer. He named it." Then she whispered conspiratorially, "He acts like it's a person." "I do not," he said as we walked down the hall toward his room. "You gave it a birthday party," she said. Grayson stopped walking for a moment. "Annual hard-drive maintenance and software upgrades do not count as a birthday party." "No," she said. "But singing 'Happy Birthday' to it does." He took a deep breath. They've obviously been through this before. "You know I was testing the new voice-recognition software." Natalie looked at me. "Birthday party.

Dijkstra once said, “Testing shows the presence, not the absence, of bugs.” In other words, a program can be proven incorrect by a test, but it cannot be proven correct. All that tests can do, after sufficient testing effort, is allow us to deem a program to be correct enough for our purposes.

Testing leads to failure, and failure leads to understanding

The earlier you catch defects, the cheaper they are to fix.

Quality must be enforced, otherwise it won't happen. We programmers must be required to write tests, otherwise we won't do it.

Trying to improve software quality by increasing the amount of testing is like trying to lose weight by weighing yourself more often. What you eat before you step onto the scale determines how much you will weigh, and the software-development techniques you use determine how many errors testing will find.

First Law You may not write production code until you have written a failing unit test. Second Law You may not write more of a unit test than is sufficient to fail, and not compiling is failing. Third Law You may not write more production code than is sufficient to pass the currently failing test.

Good testing involves balancing the need to mitigate risk against the risk of trying to gather too much information.

Scarcity brings clarity.

Suppose a developer has a conversation with a customer about details of a feature. The conversation should not be considered complete until it is expressed as a customer test.

If the discipline of requirements specification has taught us anything, it is that well-specified requirements are as formal as code and can act as executable tests of that code!

In software, when something is painful, the way to reduce the pain is to do it more frequently, not less.

No matter how elegant it is, no matter how readable and accessible, if it hath not tests, it be unclean. Dave

Indeed, there is a school of thought that any work on a branch is, in the lean sense, waste—inventory that is not being pulled into the finished product.

Automated testing is a safety net that protects the program from its programmers

Developers should be able to run all automated tests on their workstations in order to triage and fix defects.

What this team did not realize was that having dirty tests is equivalent to, if not worse than, having no tests.

If you let the tests rot, then your code will rot too. Keep your tests clean.

What makes a clean test? Three things. Readability, readability, and readability.

Most testers I’ve known are perverse enough that if you tell them the “happy path” through the application, that’s the last thing they’ll do. It should be the same with load testing.

Asking experts to do boring and repetitive, and yet technically demanding tasks is the most certain way of ensuring human error that we can think of, short of sleep deprivation, or inebriation.

There should be two tasks for a human being to perform to deploy software into a development, test, or production environment: to pick the version and environment and to press the “deploy” button.

You look concerned,” Roshaun said from behind her. Dairine scowled over her shoulder at him. “The whole universe is in danger,” she said, “and we’re not sure how to save it, assuming it can be saved. One of the Powers That Be has stuffed secret messages into my brain without telling me. And a friend of mine who happens to be my wizard’s manual is being reprogrammed with software that even these guys haven’t had time to beta test! Wow, Roshaun, why would I need to be concerned?” Roshaun glanced at the ground. Another chair appeared for him, a slight distance from Dairine’s. He lowered himself into it, stretching out his legs with a sigh. “Sarcasm,” he said, “Amusing, if ineffective.

It is unit tests that keep our code flexible, maintainable, and reusable. The reason is simple. If you have tests, you do not fear making changes to the code! Without tests every change is a possible bug.

The Joel Test 1. Do you use source control? 2. Can you make a build in one step? 3. Do you make daily builds? 4. Do you have a bug database? 5. Do you fix bugs before writing new code? 6. Do you have an up-to-date schedule? 7. Do you have a spec? 8. Do programmers have quiet working conditions? 9. Do you use the best tools money can buy? 10. Do you have testers? 11. Do new candidates write code during their interview? 12. Do you do hallway usability testing?

A system that is comprehensively tested and passes all of its tests all of the time is a testable system. That’s an obvious statement, but an important one. Systems that aren’t testable aren’t verifiable. Arguably, a system that cannot be verified should never be deployed.

most schools also focus too much on providing students with a set of predetermined skills, such as solving differential equations, writing computer code in C++, identifying chemicals in a test tube, or conversing in Chinese. Yet since we have no idea what the world and the job market will look like in 2050, we don’t really know what particular skills people will need. We might invest a lot of effort teaching kids how to write in C++ or speak Chinese, only to discover that by 2050 AI can code software far better than humans, and a new Google Translate app will enable you to conduct a conversation in almost flawless Mandarin, Cantonese, or Hakka, even though you only know how to say “Ni hao.

It is a myth that we can get systems “right the first time.” Instead, we should implement only today’s stories, then refactor and expand the system to implement new stories tomorrow. This is the essence of iterative and incremental agility. Test-driven development, refactoring, and the clean code they produce make this work at the code level.

There's never an easy answer to the question "Should we do more testing?" because information can guide risk reduction, but doesn't necessarily do so.

Testing does not improve a product; the improving is done by people fixing the bugs that testing has uncovered.

How can we hope to catch all the configuration options in an entire system? At some level we can’t, and this is at the heart of what professional testers do.

It may seem like writing tests slows down development; in fact, testing does not cost, it pays, both during development and over the system’s lifecycle.

Complexity kills. It sucks the life out of developers, it makes products difficult to plan, build, and test.” —Ray Ozzie, CTO, Microsoft Corporation

If a test in the same package needs to call a function or access a variable, we’ll make it protected or package scope.

One Domain Vocabulary A class that uses terms from multiple domains might be violating context independence, unless it’s part of a bridging layer.

Testing was never going to be fashionable, but you could hardly run a respectable software development shop with no testing effort at all.

In IT, there is nothing like ‘all of a sudden’.

Sell Specification by Example as a better way to do acceptance testing

Without tests every change is a possible bug. No

Code, without tests, is not clean. No matter how elegant it is, no matter how readable and accessible, if it hath not tests, it be unclean. Dave

Releasing software is too often an art; it should be an engineering discipline.

incremental development can be disconcerting for teams and management who aren’t used to it because it front-loads the stress in a project.

If you have bad tests, automation can help you do bad testing faster.

As for relegated/delegated responsibility to ensure organizational software licensing compliance, management is still accountable when intellectual property rights are violated. If the safeguarding responsibility is assigned to an ineffective and/or inefficient unit within an organization, IT audit should recommend an alternative arrangement after the risks are substantiated.

The moral of the story is simple: Test code is just as important as production code. It is not a second-class citizen. It requires thought, design, and care. It must be kept as clean as production code.

When is the 'look out the windshield phase' of driving? Pretty much all driving is looking out the windshield! It's not a phase. Saying 'testing takes too long' is a bit like saying 'safe driving takes too long.

In a city of twenty million like New York, there might be one or two terrorists. Maybe ten of them at the outside. 10/20,000,000 = 0.00005 percent. One twenty-thousandth of a percent. That’s pretty rare all right. Now, say you’ve got some software that can sift through all the bank records, or toll pass records, or public transit records, or phone call records in the city and catch terrorists 99 percent of the time. In a pool of twenty million people, a 99 percent accurate test will identify two hundred thousand people as being terrorists. But only ten of them are terrorists. To catch ten bad guys, you have to haul in and investigate two hundred thousand innocent people.

Often when managers say, "Testing takes too long," what they should be saying is, "Fixing the bugs in the product takes too long"—a different cost category. Make sure you're accounting for effort and time under the correct cost category.

Try to write tests that force exceptions, and then add behavior to your handler to satisfy your tests. This will cause you to build the transaction scope of the try block first and will help you maintain the transaction nature of that scope.

Clean code can be read, and enhanced by a developer other than its original author. It has unit and acceptance tests. It has meaningful names. It provides one way rather than many ways for doing one thing. It has minimal dependencies, which are explicitly defined, and provides a clear and minimal API. Code should be literate since depending on the language, not all necessary information can be expressed clearly in code alone. -Dave Thomas, founder of OTI, godfather of the Eclipse strategy

The job of a tester is to prove that the software is bug free, while it has to be the other way around: The job of a tester is to prove that the software is broken. The better testers are doing their jobs, the more bugs they manage to find and report.

Testing may convincingly demonstrate the presence of bugs, but can never demonstrate their absence."- Edsger W. Dijkstra, Computing Pioneer (1930–2002), "Programming as a discipline of mathematical nature," Am. Math. Monthly, 81 (1974), No. 6, pp. 608–12.

The job of tests, and the people that develop and runs tests, is to prevent defects, not to find them. A quality assurance organization should champion processes that build quality into the code from the start rather than test quality in later. This is not to say that verification is unnecessary. Final verification is a good idea. It’s just that finding defects should be the exception, not the rule, during verification. If verification routinely triggers test-and-fix cycles, then the development process is defective.

I learned that Bill himself has long appreciated the importance of competencies other than talent. Back in the days when he had a more direct role in hiring software programmers at Microsoft, for instance, he said he’d give applicants a programming task he knew would require hours and hours of tedious troubleshooting. This wasn’t an IQ test, or a test of programming skills. Rather, it was a test of a person’s ability to muscle through, press on, get to the finish line. Bill only hired programmers who finished what they began.

Jane easily shifts among activities—testing for discovery, pinpointing, locating, determining significance, repairing, troubleshooting, and testing to learn. As long as she continues to serve her customers, it doesn't really matter which task she performs at any given time.

I write them to improve my productivity as a programmer. Making the quality assurance department happy is just a side effect. Unit tests are highly localized. Each test class works within a single package. It tests the interfaces to other packages, but beyond that it assumes the rest just works. Functional tests are a different animal. They are written to ensure the software as a whole works. They provide quality assurance to the customer and don't care about programmer productivity. They should be developed by a different team, one who delights in finding bugs.

Robotics, however, is much more difficult. It requires a delicate interplay of mechanical engineering, perception AI, and fine-motor manipulation. These are all solvable problems, but not at nearly the speed at which pure software is being built to handle white-collar cognitive tasks. Once that robot is built, it must also be tested, sold, shipped, installed, and maintained on-site. Adjustments to the robot’s underlying algorithms can sometimes be made remotely, but any mechanical hiccups require hands-on work with the machine. All these frictions will slow down the pace of robotic automation.

Another critical element is to keep your plan 100 percent creative. Stay out of the back office. Creative work always requires noncreative work to support it: setting up software, testing tools, learning new skills, and so on. Don’t get sucked in. Never let the admin get ahead of the real work, the making and the doing.

Tests are sometimes mistaken with quality assurance. These two notions are not identical: 1) quality assurance ensures that the organization's processes are implemented and applied correctly; 2) testing identifies defects and failures, and provides information on the software and the risks associated with their release to the market

Testing gathers information about a product; it does not fix things it finds that are wrong. Testing does not improve a product; the improving is done by people fixing the bugs that testing has uncovered. Often when managers say, "Testing takes too long," what they should be saying is, "Fixing the bugs in the product takes too long"—a different cost category.

Not trusting my instincts any further, I loaded up a piece of high-end avatar dance software called Travoltra, which I’d downloaded and tested earlier that evening. The program took control of Parzival’s movements, synching them up with the music, and all four of my limbs were transformed into undulating cosine waves. Just like that, I became a dancing fool. Art3mis

Every change that is made to an application’s configuration, source code, environment, or data, triggers the creation of a new instance of the pipeline. One of the first steps in the pipeline is to create binaries and installers. The rest of the pipeline runs a series of tests on the binaries to prove that they can be released. Each test that the release candidate passes gives us more confidence that this particular combination of binary code, configuration information, environment, and data will work. If the release candidate passes all the tests, it can be released. The deployment pipeline has its foundations in the process of continuous integration and is in essence the principle of continuous integration taken to its logical conclusion. The aim of the deployment pipeline is threefold. First, it makes every part of the process of building, deploying, testing, and releasing software visible to everybody involved, aiding collaboration. Second, it improves feedback so that problems are identified, and so resolved, as early in the process as possible. Finally, it enables teams to deploy and release any version of their software to any environment at will through a fully automated process.

We now have many statistical software packages. Their power is incredible, but the pioneers of statistical inference would have mixed feelings, for they always insisted that people think before using a routine. In the old days routines took endless hours to apply, so one had to spend a lot of time thinking in order to justify using a routine. Now one enters data and presses a button. One result is that people seem to be cowed into not asking silly questions, such as: What hypothesis are you testing? What distribution is it that you say is not normal? What population are you talking about? Where did this base rate come from? Most important of all: Whose judgments do you use to calibrate scores on your questionnaires? Are those judgments generally agreed to by the qualified experts in the entire community?

Time is the strangest substance known to man. You can’t see, touch, hear, smell, taste or avoid it. Time makes you stronger-minded but weaker-bodied, gradually transforming you from blushing grape to ornery, grouching raisin. Time is the most precious thing you have, yet you’re happiest when you’re wasting it. Time will outlive you, your offspring, your offspring’s robots and your offspring’s robots’ springs.

The simplest answer is that the user had access to reality—every company builds a bubble around itself, where the products get built and tested in a more controlled environment than they get used in. This is especially true of complex software. What the early users enabled Xiaomi to see was how MIUI actually worked when real (albeit unusually technically proficient) people tried to install it on a wide variety of devices.

Denise: "So your present project is ready to pass to the testing people?" Ralph: "Absolutely." Denise: "Okay, since you're so sure it's adequately tested, I'm going to make you the following generous offer: If fewer than three bugs turn up in your component during testing, I will give you a raise. But if three or more bugs turn up during testing, you won't earn a raise this year." Ralph: "Um . . ." Denise: "Um what?" "Could I just have the component back for a few little tests I want to do?

Project managers don’t write code, they don’t test the use cases, and they’re not designing the interface. You know what a good project manager does? They are chaos-destroying machines, and each new person you bring onto your team, each dependency you create, adds hard-to-measure entropy to your team. A good project manager thrives on measuring, controlling, and crushing entropy. You did this easily when you were a team of five, but if you’re going to succeed at 105, what was done organically now needs to be done mechanically.

Life of a software engineer sucks big time during project release. Every single team member contribution is very important. At times, we have to skip breakfast, lunch and even dinner, just to make sure the given ‘TASK’ is completed. Worst thing, that’s the time we get to hear wonderful F* words. It can be on conference calls or on emails, still we have to focus and deliver the end product to a client, without any compromise on quality. Actually, every techie should be saluted. We are the reason for the evolution of Information Technology. We innovate. We love artificial intelligence. We create bots and much more. We take you closer to books. Touch and feel it without the need of carrying a paperback. We created eBook and eBook reader app: it’s basically a code of a software engineer that process the file, keeps up-to-date of your reading history, and gives you a smoother reading experience. We are amazing people. We are more than a saint of those days. Next time, when you meet a software engineer, thank him/her for whatever code he/she developed, tested, designed or whatever he/she did!

model’s blind spots reflect the judgments and priorities of its creators. While the choices in Google Maps and avionics software appear cut and dried, others are far more problematic. The value-added model in Washington, D.C., schools, to return to that example, evaluates teachers largely on the basis of students’ test scores, while ignoring how much the teachers engage the students, work on specific skills, deal with classroom management, or help students with personal and family problems. It’s overly simple, sacrificing accuracy and insight for efficiency.

Then he began writing the software that would get the microprocessor to display images on the screen. Because he could not afford to pay for computer time, he wrote the code by hand. After a couple of months he was ready to test it. "I typed a few keys on the keyboard and I was shocked! The letters were displayed on the screen." It was Sunday, June 29, 1975, a milestone for the personal computer. "It was the first time in history," Wozniak later said, "anyone had typed a character on a keyboard and seen it show up on their own computer's screen right in front of them.

Creators test ideas in low-risk experiments and quickly, creatively, and inexpensively gather insights to determine whether a product or idea will take off. By taking small risks, they avoid catastrophic mistakes. “Failure is something you know only in hindsight,” Handle cofounder and Menlo Ventures partner Shawn Carolan said. As an entrepreneur trying to solve e-mail overload with Handle, a software application, Carolan calls the twists and turns “pivots” that reveal failure only in retrospect. Creators try several approaches to find out what will work and what will not. •

In 2003, a Dutch clinical psychologist named Christof van Nimwegen began a fascinating study of computer-aided learning that a BBC writer would later call “one of the most interesting examinations of current computer use and the potential downsides of our increasing reliance on screen-based interaction with information systems.”26 Van Nimwegen had two groups of volunteers work through a tricky logic puzzle on a computer. The puzzle involved transferring colored balls between two boxes in accordance with a set of rules governing which balls could be moved at which time. One of the groups used software that had been designed to be as helpful as possible. It offered on-screen assistance during the course of solving the puzzle, providing visual cues, for instance, to highlight permitted moves. The other group used a bare-bones program, which provided no hints or other guidance. In the early stages of solving the puzzle, the group using the helpful software made correct moves more quickly than the other group, as would be expected. But as the test proceeded, the proficiency of the members of the group using the bare-bones software increased more rapidly. In the end, those using the unhelpful program were able to solve the puzzle more quickly and with fewer wrong moves. They also reached fewer impasses—states in which no further moves were possible—than did the people using the helpful software. The findings indicated, as van Nimwegen reported, that those using the unhelpful software were better able to plan ahead and plot strategy, while those using the helpful software tended to rely on simple trial and error. Often, in fact, those with the helpful software were found “to aimlessly click around” as they tried to crack the puzzle.

Some people mistakenly refer to software defects as bugs. When called bugs, they seem like pesky things that should be swatted or even ignored. This trivializes a critical problem and fosters a wrong attitude. Thus, when an engineer says there are only a few bugs left in a program, the reaction is one of relief. *Supposed, however, that we called them time bombs instead of bugs.* Would you feel the same sense of relief if a programmer told you that he had thoroughly tested a program and there were only a few time bombs left in it? Just using a different term changes your attitude entirely.

Or, suppose you want to motivate your managers to ship products on time, so you conspicuously promote each manager whose product goes out the door on schedule. All goes as planned until the situation arises in which one of your managers has a project where the testers are reporting numerous problems. Because managers who have shipped products on time have been promoted, this manager thinks, I want that promotion so I need to ship this on time, but those bug reports are getting in the way. I know what I'll do! I'll put the testers on another project until the developers have a chance to catch up.

I want to update it. I want to beta test new models. I want to break it and start all over. The way I think of it is: marriage is just a technology that was never quite future-proof. Like, it may have been a good tool in Victorian England or whatever. But for us? Now? Not so much. We have these twenty-first-century relationships running eighteenth-century software. So it’s glitchy and it crashes all the time. Typically with any technology we try to innovate and update and improve it, but with marriage we seem to refuse all progress. We’ve convinced ourselves that, actually, we like all those glitches.

Digital computers are themselves more complex than most things people build; they have very large numbers of states. This makes conceiving, describing, and testing them hard. Software systems have orders of magnitude more states than computers do. Likewise, a scaling-up of a software entity is not merely a repetition of the same elements in larger size; it is necessarily an increase in the number of different elements. In most cases, the elements interact with each other in some nonlinear fashion, and the complexity of the whole increases much more than linearly. The complexity of software is an essential property, not an accidental one.

As test documentation goes, test plans have the briefest actual lifespan of any test artifact. Early in a project, there is a push to write a test plan [...]. Indeed, there is often an insistence among project managers that a test plan must exist and that writing it is a milestone of some importance. But, once such a plan is written, it is often hard to get any of those same managers to take reviewing and updating it seriously. The test plan becomes a beloved stuffed animal in the hands of a distracted child. We want it to be there at all times. We drag it around from place to place without ever giving it any real attention. We only scream when it gets taken away.

• Do you think the outcome of testing will make your decisions for you? Business decisions cannot be made from a purely technical perspective. Certainly use information from testing to color your business decisions, but don't substitute testing for business decisions. For example, it can be a good business decision to ship a system that fails one or more tests. Conversely, it can be a bad business decision to ship a system that passes all your tests. Management has to consider other factors in addition to test results. • Conversely, is there any possible test outcome that would make you change your decisions? If not, why would you want to know the outcome, let alone pay for it?

Nintendo’s standards were exacting. “In terms of game testing they revolutionised the concept,” said Milgrom. “They said zero defects – we will not allow you to release a game that has any bugs in it whatsoever. Now zero defects was an unheard of concept in any other software or on any other gaming platform. Nintendo knew if they were going to sell it in the supermarkets and sell it to mums and dads it had to work off the shelf and had to be flawless. They didn’t want returns. We had to change our programming attitude and the way we developed games, which was brilliant. It was really hard work. If you had a bug in your final version you could miss Christmas because it would take a month for them to go through the testing of the title.

Because ease of use is the purpose, this ratio of function to conceptual complexity is the ultimate test of system design. Neither function alone nor simplicity alone defines a good design. This point is widely misunderstood. Operating System/360 is hailed by its builders as the finest ever built, because it indisputably has the most function. Function, and not simplicity, has always been the measure of excellence for its designers. On the other hand, the Time-Sharing System for the PDP-10 is hailed by its builders as the finest, because of its simplicity and the spareness of its concepts. By any measure, however, its function is not even in the same class as that of OS/360. As soon as ease of use is held up as the criterion, each of these is seen to be unbalanced, reaching for only half of the true goal.

Hiro and Y.T. have eaten a lot of junk food together in different joints all over L.A. -- doughnuts, burritos, pizza, sushi, you name it -- and all Y.T. ever talks about is her mother and the terrible job that she has with the Feds. The regimentation. The lie-detector tests. The fact that for all the work she does, she really has no idea what it is that the government is really working on. It's always been a mystery to Hiro, too, but then, that's how the government is. It was invented to do stuff that private enterprise doesn't bother with, which means that there's probably no reason for it; you never know what they're doing or why. Hackers have traditionally looked upon the government's coding sweatshops with horror and just tried to forget that all of that shit ever existed. But they have thousands of programmers. The programmers work twelve hours a day out of some twisted sense of personal loyalty. Their software-engineering techniques, while cruel and ugly, are very sophisticated. They must have been up to something.

When applying agile practices at the portfolio level, similar benefits accrue: • Demonstrable results—Every quarter or so products, or at least deployable pieces of products, are developed, implemented, tested, and accepted. Short projects deliver chunks of functionality incrementally. • Customer feedback—Each quarter product managers review results and provide feedback, and executives can view progress in terms of working products. • Better portfolio planning—Portfolio planning is more realistic because it is based on deployed whole or partial products. • Flexibility—Portfolios can be steered toward changing business goals and higher-value projects because changes are easy to incorporate at the end of each quarter. Because projects produce working products, partial value is captured rather than being lost completely as usually happens with serial projects that are terminated early. • Productivity—There is a hidden productivity improvement with agile methods from the work not done. Through constant negotiation, small projects are both eliminated and pared down.

Rich Purnell sipped coffee in the silent building. Only his cubicle illuminated the otherwise dark room. Continuing with his computations, he ran a final test on the software he'd written. It passed. With a relieved sigh, he sank back in his chair. Checking the clock on his computer, he shook his head. 3:42am. Being an astrodynamicist, Rich rarely had to work late. His job was the find the exact orbits and course corrections needed for any given mission. Usually, it was one of the first parts of a project; all the other steps being based on the orbit. But this time, things were reversed. Iris needed an orbital path, and nobody knew when it would launch. A non-Hoffman Mars-transfer isn't challenging, but it does require the exact locations of Earth and Mars. Planets move as time goes by. An orbit calculated for a specific launch date will work only for that date. Even a single day's difference would result in missing Mars entirely. So Rich had to calculate many orbits. He had a range of 25 days during which Iris might launch. He calculated one orbital path for each. He began an email to his boss. "Mike", he typed, "Attached are the orbital paths for Iris, in 1-day increments. We should start peer-review and vetting so they can be officially accepted. And you were right, I was here almost all night. It wasn't that bad. Nowhere near the pain of calculating orbits for Hermes. I know you get bored when I go in to the math, so I'll summarize: The small, constant thrust of Hermes's ion drives is much harder to deal with than the large point-thrusts of presupply probes. All 25 of the orbits take 349 days, and vary only slightly in thrust duration and angle. The fuel requirement is nearly identical for the orbits and is well within the capacity of EagleEye's booster. It's too bad. Earth and Mars are really badly positioned. Heck, it's almost easier to-" He stopped typing. Furrowing his brow, he stared in to the distance. "Hmm." he said. Grabbing his coffee cup, he went to the break room for a refill. ... "Rich", said Mike. Rich Purnell concentrated on his computer screen. His cubicle was a landfill of printouts, charts, and reference books. Empty coffee cups rested on every surface; take-out packaging littered the ground. "Rich", Mike said, more forcefully. Rich looked up. "Yeah?" "What the hell are you doing?" "Just a little side project. Something I wanted to check up on." "Well... that's fine, I guess", Mike said, "but you need to do your assigned work first. I asked for those satellite adjustments two weeks ago and you still haven't done them." "I need some supercomputer time." Rich said. "You need supercomputer time to calculate routine satellite adjustments?" "No, it's for this other thing I'm working on", Rich said. "Rich, seriously. You have to do your job." Rich thought for a moment. "Would now be a good time for a vacation?" He asked. Mike sighed. "You know what, Rich? I think now would be an ideal time for you to take a vacation." "Great!" Rich smiled. "I'll start right now." "Sure", Mike said. "Go on home. Get some rest." "Oh, I'm not going home", said Rich, returning to his calculations. Mike rubbed his eyes. "Ok, whatever. About those satellite orbits...?" "I'm on vacation", Rich said without looking up. Mike shrugged and walked away.

Bitcoin’s Test Blockchains You might be surprised to learn that there is more than one bitcoin blockchain. The “main” bitcoin blockchain, the one created by Satoshi Nakamoto on January 3rd, 2009, the one with the genesis block we studied in this chapter, is called mainnet. There are other bitcoin blockchains that are used for testing purposes: at this time testnet, segnet, and regtest. Let’s look at each in turn. Testnet — Bitcoin’s Testing Playground Testnet is the name of the test blockchain, network, and currency that is used for testing purposes. The testnet is a fully featured live P2P network, with wallets, test bitcoins (testnet coins), mining, and all the other features of mainnet. There are really only two differences: testnet coins are meant to be worthless and mining difficulty should be low enough that anyone can mine testnet coins relatively easily (keeping them worthless). Any software development that is intended for production use on bitcoin’s mainnet should first be tested on testnet with test coins. This protects both the developers from monetary losses due to bugs and the network from unintended behavior due to bugs.

our explosive growth was slowing down our pace of innovation. We were spending more time coordinating and less time building. More features meant more software, written and supported by more software engineers, so both the code base and the technical staff grew continuously. Software engineers were once free to modify any section of the entire code base to independently develop, test, and immediately deploy any new features to the website. But as the number of software engineers grew, their work overlapped and intertwined until it was often difficult for teams to complete their work independently. Each overlap created one kind of dependency, which describes something one team needs but can’t supply for itself. If my team’s work requires effort from yours—whether it’s to build something new, participate, or review—you’re one of my dependencies. Conversely, if your team needs something from mine, I’m a dependency of yours. Managing dependencies requires coordination—two or more people sitting down to hash out a solution—and coordination takes time. As Amazon grew, we realized that despite our best efforts, we were spending too much time coordinating and not enough time building. That’s because, while the growth in employees was linear, the number of their possible lines of communication grew exponentially. Regardless of what form it takes—and we’ll get into the different forms in more detail shortly—every dependency creates drag. Amazon’s growing number of dependencies delayed results, increased frustration, and disempowered teams.

Stanford University’s John Koza, who pioneered genetic programming in 1986, has used genetic algorithms to invent an antenna for NASA, create computer programs for identifying proteins, and invent general purpose electrical controllers. Twenty-three times Koza’s genetic algorithms have independently invented electronic components already patented by humans, simply by targeting the engineering specifications of the finished devices—the “fitness” criteria. For example, Koza’s algorithms invented a voltage-current conversion circuit (a device used for testing electronic equipment) that worked more accurately than the human-invented circuit designed to meet the same specs. Mysteriously, however, no one can describe how it works better—it appears to have redundant and even superfluous parts. But that’s the curious thing about genetic programming (and “evolutionary programming,” the programming family it belongs to). The code is inscrutable. The program “evolves” solutions that computer scientists cannot readily reproduce. What’s more, they can’t understand the process genetic programming followed to achieve a finished solution. A computational tool in which you understand the input and the output but not the underlying procedure is called a “black box” system. And their unknowability is a big downside for any system that uses evolutionary components. Every step toward inscrutability is a step away from accountability, or fond hopes like programming in friendliness toward humans. That doesn’t mean scientists routinely lose control of black box systems. But if cognitive architectures use them in achieving AGI, as they almost certainly will, then layers of unknowability will be at the heart of the system. Unknowability might be an unavoidable consequence of self-aware, self-improving software.

A famous British writer is revealed to be the author of an obscure mystery novel. An immigrant is granted asylum when authorities verify he wrote anonymous articles critical of his home country. And a man is convicted of murder when he’s connected to messages painted at the crime scene. The common element in these seemingly disparate cases is “forensic linguistics”—an investigative technique that helps experts determine authorship by identifying quirks in a writer’s style. Advances in computer technology can now parse text with ever-finer accuracy. Consider the recent outing of Harry Potter author J.K. Rowling as the writer of The Cuckoo’s Calling , a crime novel she published under the pen name Robert Galbraith. England’s Sunday Times , responding to an anonymous tip that Rowling was the book’s real author, hired Duquesne University’s Patrick Juola to analyze the text of Cuckoo , using software that he had spent over a decade refining. One of Juola’s tests examined sequences of adjacent words, while another zoomed in on sequences of characters; a third test tallied the most common words, while a fourth examined the author’s preference for long or short words. Juola wound up with a linguistic fingerprint—hard data on the author’s stylistic quirks. He then ran the same tests on four other books: The Casual Vacancy , Rowling’s first post-Harry Potter novel, plus three stylistically similar crime novels by other female writers. Juola concluded that Rowling was the most likely author of The Cuckoo’s Calling , since she was the only one whose writing style showed up as the closest or second-closest match in each of the tests. After consulting an Oxford linguist and receiving a concurring opinion, the newspaper confronted Rowling, who confessed. Juola completed his analysis in about half an hour. By contrast, in the early 1960s, it had taken a team of two statisticians—using what was then a state-of-the-art, high-speed computer at MIT—three years to complete a project to reveal who wrote 12 unsigned Federalist Papers. Robert Leonard, who heads the forensic linguistics program at Hofstra University, has also made a career out of determining authorship. Certified to serve as an expert witness in 13 states, he has presented evidence in cases such as that of Christopher Coleman, who was arrested in 2009 for murdering his family in Waterloo, Illinois. Leonard testified that Coleman’s writing style matched threats spray-painted at his family’s home (photo, left). Coleman was convicted and is serving a life sentence. Since forensic linguists deal in probabilities, not certainties, it is all the more essential to further refine this field of study, experts say. “There have been cases where it was my impression that the evidence on which people were freed or convicted was iffy in one way or another,” says Edward Finegan, president of the International Association of Forensic Linguists. Vanderbilt law professor Edward Cheng, an expert on the reliability of forensic evidence, says that linguistic analysis is best used when only a handful of people could have written a given text. As forensic linguistics continues to make headlines, criminals may realize the importance of choosing their words carefully. And some worry that software also can be used to obscure distinctive written styles. “Anything that you can identify to analyze,” says Juola, “I can identify and try to hide.

Agile is quite simple. The most popular Agile approach, Scrum, has just three roles, a handful of activities, and one major artifact: running tested software. That doesn’t mean Agile is easy. It’s still hard to decide what product would be desirable, and it’s still hard to write software that does what is asked for. It is, however, quite simple. Simplicity is the essence of what makes up Agility, as we described in our chapter on Value. So if Agile is simple, what about “Scaled Agile”?

Generative testing is an approach to testing software that was made popular by the QuickCheck library. Originally written in Haskell and since ported to several other programming languages (Ruby, Python, C, C++, Objective-C, Smalltalk, Java, JavaScript, Erlang, Scala, Clojure...), the QuickCheck library allows the developer to separate test logic from the generation of test cases. This means that, as developers, we can spend less time instructing the compiler how to test our code, and focus instead on what properties we expect our code to have.

Folk wisdom in software development teaches that interfaces shouldn't be unduly influenced by implementations. Writing a test first is a concrete way to achieve this separation.