Reliability Engineering Quotes

If it hurts, do it more frequently, and bring the pain forward.

I'm not supposed to be the reliable one here! I'm supposed to be the one who stabs people for money, and this lovesick bullshit of yours is seriously cramping my style!

...every new genetically engineered plant is a unique event in nature, bringing its own set of genetic contingencies. This means that the reliability or safety of one genetically modified plant doesn't necessarily guarantee the reliability or safety of the next.

Never forget that the word that best describes reliable science is not consensus, but engineering.

Without the low operating costs, high efficiency, high reliability, and great durability of diesel engines, it would have been impossible to reach the extent of globalization that now defines the modern economy.

Making all the components highly reliable will not necessarily make the system safe.

When a team must allocate a disproportionate amount of time to resolving tickets at the cost of spending time improving the service, scalability and reliability suffer.

team size should not scale directly with service growth.

Releasing software is too often an art; it should be an engineering discipline.

generally speaking, no one should be directly logging in to a database.

Hope is not a strategy.

Leibniz had little engineering skill and did not surround himself with those who did. So, like many great theorists who lacked practical collaborators, he was unable to produce reliably working versions of his device.

At this risk of being redundant, everything must be checked in to code. This includes the following: DB object migrations Triggers Procedures and functions Views Configurations Sample datasets for functionality Data cleanup scripts

The train swung around the curve, the engine puffing with short, heavy blasts, and they passed smoothly from sight that way, with that quality about them of shabby and timeless patience, of static serenity: that blending of childlike and ready incompetence and paradoxical reliability that tends and protects them it loves out of all reason and robs them steadily and evades responsibility and obligations by means too barefaced to be called subterfuge even and is taken in theft or evasion with only that frank and spontaneous admiration for the victor which a gentleman feels for anyone who beats him in a fair contest, and withal a fond and unflagging tolerance for whitefolk's vagaries like that of a grandparent for unpredictable and troublesome children, which I had forgotten.

Bill Baker, a distinguished engineer at Microsoft, quipped that we used to treat servers like pets: “You name them and when they get sick, you nurse them back to health. [Now] servers are [treated] like cattle. You number them and when they get sick, you shoot them.

As electrical energy can create mechanical vibrations (perceived as sound by the human ear), so in turn can mechanical vibrations create electrical energy, such as the previously mentioned ball lightning. It could be theorized, therefore, that with the Earth being a source for mechanical vibration, or sound, and the vibrations being of a usable amplitude and frequency, then the Earth's vibrations could be a source of energy that we could tap into. Moreover, if we were to discover that a structure with a certain shape, such as a pyramid, was able to effectively act as a resonator for the vibrations coming from within the Earth, then we would have a reliable and inexpensive source of energy.

Things are getting out of hand: Tony finds herself channeling this opinion at least once a day. The crazed weather. The vicious, hate-filled politics. The myriad glass high-rises going up like 3-D mirrors, or siege engines. The municipal garbage collection: Who can keep all those different-coloured bins straight? Where to put the clear plastic food containers, and why isn't the little number on the bottom a reliable guide? And the vampires. You used to know where you stood with them--smelly, evil,undead--but now there are virtuous vampires and disreputable vampires, and sexy vampires and glittery vampires, and none of the old rules about them are true any more. Once you could depend on garlic, and on the rising sun, and on crucifixes. You could get rid of the vampires once and for all. But not any more.

In the early 1980s, managers at the National Aeronautics and Space Administration (NASA) estimated that the flights would be 99.999 percent reliable, which represents a failure rate of only 1 in 100,000. According to the physicist Richard Feynman, who was a member of the commission that investigated the January 1986 Challenger accident, in which the shuttle broke apart shortly into its flight, killing all seven astronauts on board, this “would imply that one could put a Shuttle up each day for 300 years expecting to lose only one.” He wondered, “What is the cause of management’s fantastic faith in the machinery?” Engineers, who were more familiar with the shuttle itself and with machines in general, predicted only a 99 percent success rate, or a failure every 100 launches. A range safety officer, who personally observed test firings during the developmental phase of the rocket motors, expected a failure rate of 1 in 25. The Challenger accident proved that estimate to be the actual failure rate, giving a success rate of 96 percent after exactly 25 launchings.

When his troops reached the rocket camp, they found almost everything of value gone and Stalin was reportedly furious. “This is absolutely intolerable,” he said, according to reliable defectors. “We defeated Nazi armies; we occupied Berlin and Peenemünde, but the Americans got the rocket engineers. What could be more revolting and more inexcusable? How and why was this allowed to happen?” In a way, the Red Army’s race toward Peenemünde was symbolic: It was, without anyone knowing it, the beginning of the race for outer space, or what Winston Churchill once called “the wizard war.

It’s hard to blame Representative Petri for missing the point. The value of studying fireflies is endlessly surprising. For example, before 1994, Internet engineers were vexed by spontaneous pulsations in the traffic between computers called routers, until they realized that the machines were behaving like fireflies, exchanging periodic messages that were inadvertently synchronizing them. Once the cause was identified, it became clear how to relieve the congestion. Electrical engineers devised a decentralized architecture for clocking computer circuits more efficiently, by mimicking the fireflies’ strategy for achieving synchrony at low cost and high reliability.

What was happily proved by this early revolution is something that we perhaps need to be reminded of again today: that neither exact science nor engineering is proof against the irrationality of those that operate the system. Above all, that the strongest and most efficient of megamachines can be overthrown, that human errors are not immortal. The collapse of the Pyramid Age proved that the megamachine exists on a basis of human beliefs, which may crumble, of human decisions, which may prove fallible, and human consent, which, when the magic becomes discredited, may be withheld. The human parts that composed the megamachine were by nature mechanically imperfect: never wholly reliable. Until real machines of wood and metal could be manufactured in sufficient quantity to take the place of most of the human components, the megamachine would remain vulnerable.

The foundation of our thinking is the Theory of Jobs to Be Done, which focuses on deeply understanding your customers’ struggle for progress and then creating the right solution and attendant set of experiences to ensure you solve your customers’ jobs well, every time. “Theory” may conjure up images of ivory tower musings, but I assure you that it is the most practical and useful business tool we can offer you. Good theory helps us understand “how” and “why.” It helps us make sense of how the world works and predict the consequences of our decisions and our actions. Jobs Theory5, we believe, can move companies beyond hoping that correlation is enough to the causal mechanism of successful innovation. Innovation may never be a perfect science, but that’s not the point. We have the ability to make innovation a reliable engine for growth, an engine based on a clear understanding of causality, rather than simply casting seeds in the hopes of one day harvesting some fruit.

Leonid Gavrilov, a researcher at the University of Chicago, argues that human beings fail the way all complex systems fail: randomly and gradually. As engineers have long recognized, simple devices typically do not age. They function reliably until a critical component fails, and the whole thing dies in an instant. A windup toy, for example, works smoothly until a gear rusts or a spring breaks, and then it doesn’t work at all. But complex systems—power plants, say—have to survive and function despite having thousands of critical, potentially fragile components. Engineers therefore design these machines with multiple layers of redundancy: with backup systems, and backup systems for the backup systems. The backups may not be as efficient as the first-line components, but they allow the machine to keep going even as damage accumulates. Gavrilov argues that, within the parameters established by our genes, that’s exactly how human beings appear to work. We have an extra kidney, an extra lung, an extra gonad, extra teeth.

Let the center be your home: To be centered is considered desirable; when they feel distracted or scattered, people often say, “I lost my center.” But if there is no person inside your head, if the ego’s sense of I, me, mine is illusory, where’s the center? Paradoxically, the center is everywhere. It is the open space that has no boundaries. Instead of thinking of your center as a defined spot—the way people point to their hearts as the seat of the soul—be at the center of experience. Experience isn’t a place; it’s a focus of attention. You can live there, at the still point around which everything revolves. To be off center is to lose focus, to look away from experience or block it out. To be centered is like saying “I want to find my home in creation.” You relax into the rhythm of your own life, which sets the stage for meeting yourself at a deeper level. You can’t summon the silent witness, but you can place yourself close to it by refusing to get lost in your own creation. When I find myself being overshadowed by anything, I can fall back on a few simple steps: • I say to myself, “This situation may be shaking me, but I am more than any situation.” • I take a deep breath and focus my attention on whatever my body is feeling. • I step back and see myself as another person would see me (preferably the person whom I am resisting or reacting to). • I realize that my emotions are not reliable guides to what is permanent and real. They are momentary reactions, and most likely they are born of habit. • If I am about to burst out with uncontrollable reactions, I walk away. As you can see, I don’t try to feel better, to be more positive, to come from love, or to change the state I’m in. We are all framed by personalities and driven by egos. Ego personalities are trained by habit and by the past; they run along like self-propelled engines. If you can observe the mechanism at work without getting wrapped up in it, you will find that you possess a second perspective, one that is always calm, alert, detached, tuned in but not overshadowed. That second place is your center. It isn’t a place at all but a close encounter with the silent witness.

Professional Bio of Shahin Shardi, P.Eng. Materials Engineer Welding and Pressure Equipment Inspector, QA/QC Specialist Shahin Shardi is a Materials Engineer with experience in integrity management, inspection of pressure equipment, quality control/assurance of large scale oil and gas projects and welding inspection. He stared his career in trades which helped him understand fundamentals of operation of a construction site and execution of large scale projects. This invaluable experience provided him with boots on the ground perspective of requirements of running a successful project and job site. After obtaining an engineering degree from university of British Columbia, he started a career in asset integrity management for oil and gas facilities and inspection of pressure equipment in Alberta, Canada. He has been involved with numerus maintenance shutdowns at various facilities providing engineering support to the maintenance, operations and project personnel regarding selection, repair, maintenance, troubleshooting and long term reliability of equipment. In addition he has extensive experience in area of quality control and assurance of new construction activities in oil and gas industry. He has performed Owner’s Inspector and welding inspector roles in this area. Shahin has extensively applied industry codes of constructions such as ASME Pressure Vessel Code (ASME VIII), Welding (ASME IX), Process Piping (ASME B31.3), Pipe Flanges (ASME B16.5) and various pressure equipment codes and standards. Familiarity with NDT techniques like magnetic particle, liquid penetrant, eddy current, ultrasonic and digital radiography is another valuable knowledge base gained during various projects. Some of his industry certificates are CWB Level 2 Certified Welding Inspector, API 510 Pressure Vessel Inspector, Alberta ABSA In-Service Pressure Vessel Inspector and Saskatchewan TSASK Pressure Equipment Inspector. Shahin is a professional member of Association of Professional Engineers and Geoscientists of Alberta.

I am assured that this is a true story. A man calls up his computer helpline complaining that the cupholder on his personal computer has snapped off, and he wants to know how to get it fixed. “Cupholder?” says the computer helpline person, puzzled. “I’m sorry, sir, but I’m confused. Did you buy this cupholder at a computer show or receive it as a special promotion?” “No, it came as part of the standard equipment on my computer.” “But our computers don’t come with cupholders.” “Well, pardon me, friend, but they do,” says the man a little hotly. “I’m looking at mine right now. You push a button on the base of the unit and it slides right out.” The man, it transpired, had been using the CD drawer on his computer to hold his coffee cup. I bring this up here by way of introducing our topic this week: cupholders. Cupholders are taking over the world. It would be almost impossible to exaggerate the importance of cupholders in automotive circles these days. The New York Times recently ran a long article in which it tested a dozen family cars. It rated each of them for ten important features, among them engine size, trunk space, handling, quality of suspension, and, yes, number of cupholders. A car dealer acquaintance of ours tells us that they are one of the first things people remark on, ask about, or play with when they come to look at a car. People buy cars on the basis of cupholders. Nearly all car advertisements note the number of cupholders prominently in the text. Some cars, like the newest model of the Dodge Caravan, come with as many as seventeen cupholders. The largest Caravan holds seven passengers. Now you don’t have to be a nuclear physicist, or even wide awake, to work out that that is 2.43 cupholders per passenger. Why, you may reasonably wonder, would each passenger in a vehicle need 2.43 cupholders? Good question. Americans, it is true, consume positively staggering volumes of fluids. One of our local gas stations, I am reliably informed, sells a flavored confection called a Slurpee in containers up to 60 ounces in size. But even if every member of the family had a Slurpee and a personal bottle of

As strangeness becomes the new normal, your past experiences, as well as the past experiences of the whole of humanity, will become less reliable guides. Humans as individuals and humankind as a whole will increasingly have to deal with things nobody ever encountered before, such as super-intelligent machines, engineered bodies, algorithms that can manipulate your emotions with uncanny precision, rapid man-made climate cataclysms and the need to change your profession every decade. What is the right thing to do when confronting a completely unprecedented situation? How should you act when you are flooded by enormous amounts of information and there is absolutely no way you can absorb and analyse it all? How to live in a world where profound uncertainty is not a bug, but a feature? To survive and flourish in such a world, you will need a lot of mental flexibility and great reserves of emotional balance. You will have to repeatedly let go of some of what you know best, and feel at home with the unknown. Unfortunately, teaching kids to embrace the unknown and to keep their mental balance is far more difficult than teaching them an equation in physics or the causes of the First World War. You cannot learn resilience by reading a book or listening to a lecture. The teachers themselves usually lack the mental flexibility that the twenty-first century demands, for they themselves are the product of the old educational system. The Industrial Revolution has bequeathed us the production-line theory of education. In the middle of town there is a large concrete building divided into many identical rooms, each room equipped with rows of desks and chairs. At the sound of a bell, you go to one of these rooms together with thirty other kids who were all born the same year as you. Every hour some grown-up walks in, and starts talking. They are all paid to do so by the government. One of them tells you about the shape of the earth, another tells you about the human past, and a third tells you about the human body. It is easy to laugh at this model, and almost everybody agrees that no matter its past achievements, it is now bankrupt. But so far we haven’t created a viable alternative. Certainly not a scaleable alternative that can be implemented in rural Mexico rather than just in upmarket California suburbs.

Now, with their miniature robot army, three Harvard University researchers have upped the ante, assembling a massive swarm of simple, three-legged robots that can work as a team to assemble into different shapes on command. The advance, reported Thursday in the journal ­Science, is a feat of “engineering majesty,’’ said James ­McLurkin, director of the Multi-Robot Systems Lab at Rice University, who was not ­involved in the research. “Building 1,000 robots is hard,’’ McLurkin said. “Getting 1,000 robots to work together reliably is — how’d they say it in Boston? — ‘wicked hard.’ ’’ The technology is still in the early stages. These simple ­robots, which each weigh about as much as three nickels and cost $14 in parts, cannot build a skyscraper or clean up an oil spill. But they surmount several major problems in robotics, McLurkin said. The software the researchers designed allows individual ­robots to act on their own, using only information from their neighbors to achieve goals that dwarf their thumb-sized bodies.

The most common accident causality models assume that accidents are caused by component failure and that making system components highly reliable or planning for their failure will prevent accidents. While this assumption is true in the relatively simple electromechanical systems of the past, it is no longer true for the types of complex sociotechnical systems we are building today.

100% is the wrong reliability target for basically everything (pacemakers and anti-lock brakes being notable exceptions).

The launch cycle at Internet companies is markedly different. Launches and rapid iterations are far easier because new features can be rolled out on the server side, rather than requiring software rollout on individual customer workstations.

The Role of the Launch Coordination Engineer Our Launch Coordination Engineering team is composed of Launch Coordination Engineers (LCEs), who are either hired directly into this role, or are SREs with hands-on experience running Google services. LCEs are held to the same technical requirements as any other SRE, and are also expected to have strong communication and leadership skills

Any organization that aspires to be serious about running an effective SRE arm needs to consider training. Teaching SREs how to think in a complicated and fast-changing environment with a well-thought-out and well-executed training program has the promise of instilling best practices within a new hire’s first few weeks or months that otherwise would take months or years to accumulate.

Hiring SREs well is critical to having a high-functioning reliability organization, as explored in “Hiring Site Reliability Engineers” [Jon15]. Google’s hiring practices have been detailed in texts like Work Rules! [Boc15],1 but hiring SREs has its own set of particularities. Even by Google’s overall standards, SRE candidates are difficult to find and even harder to interview effectively.

Making the jump from a previous company or university, while changing job roles (from traditional software engineer or traditional systems administrator) to this nebulous Site Reliability Engineer role is often enough to knock students’ confidence down several times. For more introspective personalities (especially regarding questions #2 and #3), the uncertainties incurred by nebulous or less-than-clear answers can lead to slower development or retention problems.

Which backends of this server are considered “in the critical path,” and why? What aspects of this server could be simplified or automated? Where do you think the first bottleneck is in this architecture? If that bottleneck were to be saturated, what steps could you take to alleviate it?

In the course of their jobs, they will come across systems they’ve never seen before, so they need to have strong reverse engineering skills.

At scale, there will be anomalies that are hard to detect, so they’ll need the ability to think statistically, rather than procedurally, to uncloak problems.

When standard operating procedures break down, they’ll need to be able to improvise fully.

Those who cannot remember the past are condemned to repeat it.” George Santayana, philosopher and essayist

It’s important to establish credibility by delivering some product of value in a reasonable amount of time. Your first round of products should aim for relatively straightforward and achievable targets — ones without controversy or existing solutions. We

Production meetings are a special kind of meeting where an SRE team carefully articulates to itself — and to its invitees — the state of the service(s) in their charge, so as to increase general awareness among everyone who cares, and to improve the operation of the service(s).

Connecting the performance of the service with design decisions in a regular meeting is an immensely powerful feedback loop.

There’s a lot of evidence suggesting that diverse teams are simply better teams [Nel14]

I thought the submarine environment would be a useful analogy for the space station in a number of ways, and I especially wanted my colleagues to get an up-close look at how the Navy deals with CO2. What we learned on that trip was illuminating: the Navy has their submarines turn on their air scrubbers when the CO2 concentration rises above two millimeters of mercury, even though the scrubbers are noisy and risk giving away the submarine’s location. By comparison, the international agreement on ISS says the CO2 is acceptable up to six millimeters of mercury! The submarine’s chief engineering officer explained to us that the symptoms of high CO2 posed a threat to their work, so keeping that level low was a priority. I felt that NASA should be thinking of it the same way. When I prepared for my first flight on the ISS, I got acquainted with a new carbon dioxide removal system. The lithium hydroxide cartridges were foolproof and reliable, but that system depended on cartridges that were to be thrown away after use—not very practical, since hundreds of cartridges would be required to get through a single six-month mission. So instead we now have a device called the carbon dioxide removal assembly, or CDRA, pronounced “seedra,” and it has become the bane of my existence. There are two of them—one in the U.S. lab and one in Node 3. Each weighs about five hundred pounds and looks something like a car engine. Covered in greenish brown insulation, the Seedra is a collection of electronic boxes, sensors, heaters, valves, fans, and absorbent beds. The absorbent beds use a zeolite crystal to separate the CO2 from the air, after which the lab Seedra dumps the CO2 out into space through a vacuum valve, while the Node 3 Seedra combines oxygen drawn from the CO2 with leftover hydrogen from our oxygen-generating system in a device called Sabatier. The result is water—which we drink—and methane, which is also vented overboard.

the SRE Way in mind: thoroughness and dedication, belief in the value of preparation and documentation, and an awareness of what could go wrong, coupled with a strong desire to prevent it.

However, a real implementation may still have to include code to handle the case where something happens that was assumed to be impossible, even if that handling boils down to printf("Sucks to be you") and exit(666) — i.e., letting a human operator clean up the mess [93]. (This is arguably the difference between computer science and software engineering.)

Don’t be afraid to provide white glove customer support for early adopters to help them through the onboarding process. Sometimes automation also entails a host of emotional concerns, such as fear that someone’s job will be replaced by a shell script. By working one-on-one with early users, you can address those fears personally, and demonstrate that rather than owning the toil of performing a tedious task manually, the team instead owns the configurations, processes, and ultimate results of their technical work. Later adopters are convinced by the happy examples of early adopters.

To this end, Google always strives to staff its SRE teams with a mix of engineers with traditional software development experience and engineers with systems engineering experience.

In order to remain reliable and to avoid scaling the number of SREs supporting a service linearly, the production environment has to run mostly unattended. To remain unattended, the environment must be resilient against minor faults.

And, as we all know, culture beats strategy every time: [Mer11]

CONFUSION 1: WHAT DOES YOUR CUSTOMER REALLY WANT? Your Customers aren’t just people; they’re very specific kinds of people. Let me share with you the six categories of Customers as seen from an E-Myth marketing perspective: (1) Tactile Customers; (2) Neutral Customers; (3) Withdrawal Customers; (4) Experimental Customers; (5) Transitional Customers; and (6) Traditional Customers. Your entire marketing strategy must be based on which types of Customers you are dealing with. Each of the six customer types buys products and services for very different, and identifiable, reasons. And these are: 1. Tactile Customers get their major gratification from interacting with other people. 2. Neutral Customers get their major gratification from interacting with inanimate objects (a computer, a car, information). 3. Withdrawal Customers get their major gratification from interacting with ideas (thoughts, concepts, stories). 4. Experimental Customers rationalize their buying decisions by perceiving that what they bought is new, revolutionary, and innovative. 5. Transitional Customers rationalize their buying decisions by perceiving that what they bought is dependable and reliable. 6. Traditional Customers rationalize their buying decisions by perceiving that what they bought is cost-effective, a good deal, and worth the money. In short: 1. If your Customer is Tactile, you have to emphasize the people of your business. 2. If your Customer is Neutral, you have to emphasize the technology of your business. 3. If your Customer is a Withdrawal Customer, you have to emphasize the idea of your business. 4. If your Customer is an Experimental Customer, you have to emphasize the uniqueness of your business. 5. If your Customer is Transitional, you have to emphasize the dependability of your business. 6. If your Customer is Traditional, you have to talk about the financial competitiveness of your business. Additionally, what your Customers want is determined by who they are. Who they are is regularly demonstrated by what they do. Think about the Customers with whom you do business. Ask yourself: In which of the categories would I place them? What do they do for a living? For example: If they are mechanical engineers, they are probably Neutral Customers. If they are cardiologists, they are probably Tactile. If they are software engineers, they are probably Experimental. If they are accountants, they are probably Traditional. But don’t take my word for it. Make your own analysis.

Electric light, the first reliable internal combustion engine, and wireless transmission were all invented within the same three-month period at the end of 1879. Within

One of the most significant things they did to help change the outcomes of deployments was to have all Facebook engineers, engineering managers, and architects rotate through on-call duty for the services they built. By

We cannot achieve deployments on demand if each of our production code deployments take weeks or months to perform (i.e., each deployment requires 1,300 manual, error-prone steps involving up to three hundred engineers). The countermeasure is to automate our deployments as much as possible, with the goal of being completely automated so they can be done self-service by any developer.

Chuck Rossi, Director of Release Engineering at Facebook, described, “All the code supporting every feature we’re planning to launch over the next six months has already been deployed onto our production servers. All we need to do is turn it on.

Over-alerting causes Operations engineers to be woken up in the middle of the night for protracted periods of time, even when there are few actions that they can appropriately take. The

Between 1931 and 1946, Pan American Airways had 28 flying boats known as “Clippers,” These four radial engine aircraft were S-40’s and 42’s built in 1934, later replaced by Boeing 314 Clippers, that became the familiar symbol of the company. Following the war, Pan American Airways flew land based airliners such as the Boeing 377 Stratocruiser, developed from the C-97, Stratofreighter, and a military derivative of the B-29 Superfortress, used as a troop transport, and the DC-4 series, converted from the blueprints of the C-54 Skymaster. Both of these airliners were originally developed for the United States Army Air Corps, during World War II. On January 1950 Pan American Airways Corporation adopted the name it had been unofficially called since 1943, and formally became “Pan American World Airways, Inc.” That September Pan American bought out American Airlines’ overseas division and simultaneously placed an order for 45 DC-6Bs, replacing their DC-4’s. Throughout Pan-American was known simply as Pan-Am. The Douglas DC-6 is a four engine “Double Wasp” radial piston-powered airliner manufactured for long flights. It was built by the Douglas Aircraft Company from 1946 until 1958. More than 700 were built between those years and some are still flying today. The rugged, reliable DC-6B, was regarded as the ultimate piston-engine airliner, from the perspective of having excellent handling qualities and relatively economical operations.

Another way we can enable more market-oriented outcomes is by enabling product teams to become more self-sufficient by embedding Operations engineers within them, thus reducing their reliance on centralized Operations.

When Dev teams had problems with testing or deployment, they needed more than just technology or environments. What they also needed was help and coaching. At first, we embedded Ops engineers and architects into each of the Dev teams, but there simply weren’t enough Ops engineers to cover that many teams. We were able to help more teams with what we called an Ops liaison model and with fewer people.

Vimes knew that he had lost. He had lost as soon as Sybil was involved, because she was always a reliable siege engine against the walls of his defenses.

Site Reliability Engineering is an approach to the operation and improvement of software applications pioneered by Google to deal with their global, multi-million-user systems. If adopted in full, SRE is significantly different from IT operations of the past, due to its focus on the “error budget” (namely defining what is an acceptable amount of downtime) and the ability of SRE teams to push back on poor software.

Control system overlays must be faster and more reliable than the underlying systems being controlled. The Nyquist-Shannon Sampling Theorem, first introduced in 1928, explains why. A receiver (sensor) must sample at least twice the rate of the sender (the thing being monitored and controlled) to accurately measure and control a system. This theorem forms the basis of all things digital, including telecommunications, medical imaging systems, astronomy, and more. In reality, to control a complex engineered or biological system, the receiver and controller must be much faster to maintain resilience and agility. This has stark implications for top-down management. For instance, if reports are generated and reviewed once a week, they can be used to control (manage) only situations that change no faster or more frequently than every two weeks. Anything faster moving may not be detected or is not controllable. This explains why exemplary organizations are typically characterized by overlays of people in supportive roles that are uncharacteristic of their lower-performing peers. That is not “overhead” but absolutely necessary bandwidth for sustaining high performance of fast-moving, complex, dynamic systems.

Count on Wright Car Care for top-notch auto repair services in Dunwoody. Our skilled technicians will provide routine maintenance, computer diagnostics, engine performance upgrades, transmission service, A/C and heating repairs, full undercar service, electrical system diagnostics, and reliable roadside assistance. We use original mechanical parts and state-of-the-art technology for accurate diagnosis. Whether it’s a minor inspection or a major upgrade, we’ll make sure your car runs smoothly. Contact us at 770-451-6789 for comprehensive and effective auto repair solutions in Dunwoody that support our commitment to customer satisfaction.

In software development, engineers are often pressured to deliver features instead of fixing defects, addressing reliability issues, or working on internal improvement projects. As a result, so called “technical debt”§§ adds up, leading to more complex problems that are increasingly difficult to solve. Consequently, Marty Cagan, a product expert who has trained generations of product leaders on building software products that customers love, stresses the importance for product and engineering leaders to allocate at least 20% of engineering’s time to proactively fix issues before they snowball into catastrophic ones, such as having to “rewrite the entire codebase from scratch.

Reliable drilling contractors in Northern Ireland. Expert services for residential, commercial, & industrial projects. Contact us today.

Excellence in Statistics: Rigor Statisticians are specialists in coming to conclusions beyond your data safely—they are your best protection against fooling yourself in an uncertain world. To them, inferring something sloppily is a greater sin than leaving your mind a blank slate, so expect a good statistician to put the brakes on your exuberance. They care deeply about whether the methods applied are right for the problem and they agonize over which inferences are valid from the information at hand. The result? A perspective that helps leaders make important decisions in a risk-controlled manner. In other words, they use data to minimize the chance that you’ll come to an unwise conclusion. Excellence in Machine Learning: Performance You might be an applied machine-learning/AI engineer if your response to “I bet you couldn’t build a model that passes testing at 99.99999% accuracy” is “Watch me.” With the coding chops to build both prototypes and production systems that work and the stubborn resilience to fail every hour for several years if that’s what it takes, machine-learning specialists know that they won’t find the perfect solution in a textbook. Instead, they’ll be engaged in a marathon of trial and error. Having great intuition for how long it’ll take them to try each new option is a huge plus and is more valuable than an intimate knowledge of how the algorithms work (though it’s nice to have both). Performance means more than clearing a metric—it also means reliable, scalable, and easy-to-maintain models that perform well in production. Engineering excellence is a must. The result? A system that automates a tricky task well enough to pass your statistician’s strict testing bar and deliver the audacious performance a business leader demands. Wide Versus Deep What the previous two roles have in common is that they both provide high-effort solutions to specific problems. If the problems they tackle aren’t worth solving, you end up wasting their time and your money. A frequent lament among business leaders is, “Our data science group is useless.” And the problem usually lies in an absence of analytics expertise. Statisticians and machine-learning engineers are narrow-and-deep workers—the shape of a rabbit hole, incidentally—so it’s really important to point them at problems that deserve the effort. If your experts are carefully solving the wrong problems, your investment in data science will suffer low returns. To ensure that you can make good use of narrow-and-deep experts, you either need to be sure you already have the right problem or you need a wide-and-shallow approach to finding one.

Experimentation is the only reliable way of testing, so we measure the effect of engineers’ texts on customer satisfaction against a control group who receive no such early warning.

The greatest predictor of social pathology in children is fatherlessness, greater even than poverty. In his book Fatherless Generation (Zondervan) John Sowers claims, “The most reliable predictor for gang activity and youth violence is neither social class nor race or education but fatherlessness.” In Fatherless America (Basic Books) David Blankenhorn says, “It is no exaggeration to say that fatherlessness is the most harmful demographic trend of this generation. It is the engine driving our most urgent social problems”. I am convinced that the damage to humanity caused by the epidemic of unfathered men and women is far greater than the damage caused by war and disease combined.

By 2006, CustomerSat’s software had become complex and unwieldy. Too many quick fixes by too-junior software developers for too many years had caused reliability problems, performance bottlenecks, and security holes. We had lost a previous head of engineering, in part, because these problems seemed insurmountable.

While by definition, starship engine rooms should have been predictable, uneventful places that operated according to the reliable mathematics of warp physics, she’d come to believe that, more often than not, they were in fact the nexi of entropy. Order battled chaos in these places with an almost dependable regularity. And engineers, she secretly suspected, functioned as avatars of both these forces, keeping them carefully balanced so that neither overwhelmed the other. Thus, warp drive worked, but the best engineers could still find a new wrinkle in the laws of physics when circumstances required it. Bhatnagar

When an engineer with years of familiarity in a problem space begins designing a product, it’s easy to imagine a utopian end-state for the work. However, it’s important to differentiate aspirational goals of the product from minimum success criteria (or Minimum Viable Product). Projects can lose credibility and fail by promising too much,

The deal [between product owners and] engineering goes like this: Product management takes 20% of the team’s capacity right off the top and gives this to engineering to spend as they see fit. They might use it to rewrite, re-architect, or re-factor problematic parts of the code base...whatever they believe is necessary to avoid ever having to come to the team and say, ‘we need to stop and rewrite [all our code].’ If you’re in really bad shape today, you might need to make this 30% or even more of the resources. However, I get nervous when I find teams that think they can get away with much less than 20%.

Putting alerts into email and hoping that someone will read all of them and notice the important ones is the moral equivalent of piping them to /dev/null: they will eventually be ignored.

Imagine trying to jerry-rig a Volkswagen Beetle to travel at speeds of 150 miles per hour. In 1933, Adolf Hitler commissioned Dr. Ferdinand Porsche to develop a cheap car that could get 40 miles per gallon of gas and provide a reliable form of transportation for the average German family. The result was the VW Beetle. This history, Hitler’s plan, places constraints on the ways we can modify the Beetle today; the engineering can be tweaked only so far before major problems arise and the car reaches its limit. In many ways, we humans are the fish equivalent of a hot-rod Beetle. Take the body plan of a fish, dress it up to be a mammal, then tweak and twist that mammal until it walks on two legs, talks, thinks, and has superfine control of its fingers—and you have a recipe for problems. We can dress up a fish only so much without paying a price. In a perfectly designed world—one with no history—we would not have to suffer everything from hemorrhoids to cancer.

There is no doubt that 'force multipliers' - squad automatic weapons - have changed the character of warfare once again, just as their predecessors did during the First World War, if perhaps not to quite the same degree. In the immediate future it seems that most armies will be using some form of 5.56mm machine-gun at squad level, be it a box-fed LSW or belt-fed SAW. If there is a cloud on the horizon where modern light machine-guns are concerned it is that they are not powerful enough for long-range work, or for penetrating cover and light armour. Nevertheless, the new generation of light machine-guns will remain in use well into the next century, not least because they are popular with the soldiers who operate them, the machine-gunners. Likewise, there will still be a place for the heavier GPMG, which does have the 'punch' that the LSW lacks. Machine-guns themselves have become lighter, and their operating principles both more secure and more efficient; the ammunition they use has shrunk to a quarter of its original size and become almost 100 percent reliable. The one important thing which has not changed dramatically is the human component; the attitude with which man faces the prospect of death in battle, and how he prepares himself to face that possibility quite deliberately, for it was the original invention of the machine-gun which reformed that. More than any other single 'advance' in weapons technology, the machine-gun allowed an individual (or actually, a small team of men) to dominate a sector of the battlefield. They had an inhuman advantage which simply had to be exploited if they were to be on the winning side, whether their opponents were Zulus, Sioux, or Dervishes, or other industrialized nations to be beaten into last place in the race toward economic supremacy. Whether the machine-gun has been as important, in any sense at all of the word, as it near-contemporary, the internal combustion engine - or even, date one say it, the bicycle or sewing machine - is still to be decided, but there is one clear, irrefutable fact connected with its short history: it has killed tens of millions of men, women and children and blighted the lives of tens of millions more.

Throughout the heart of the Rockies, tourism and recreation have either replaced resource extraction industries as the major sources of revenue and employment or are on the verge of doing so. In particular, national parks and other reserves that dot the region are proving to be among its most reliable economic engines both for local communities and for the states and provinces that host the protected areas. The aim of connecting those dots for ecological reasons goes hand in hand with ensuring long-term financial stability and prosperity.

Because our goal is to enable small teams of developers to independently develop, test, and deploy value to customers quickly and reliably, this is where we want our constraint to be. High performers, regardless of whether an engineer is in Development, QA, Ops, or Infosec, state that their goal is to help maximize developer productivity.

M113 Family of Vehicles Mission Provide a highly mobile, survivable, and reliable tracked-vehicle platform that is able to keep pace with Abrams- and Bradley-equipped units and that is adaptable to a wide range of current and future battlefield tasks through the integration of specialised mission modules at minimum operational and support cost. Entered Army Service 1960 Description and Specifications After more than four decades, the M113 family of vehicles (FOV) is still in service in the U.S. Army (and in many foreign armies). The original M113 Armoured Personnel Carrier (APC) helped to revolutionise mobile military operations. These vehicles carried 11 soldiers plus a driver and track commander under armour protection across hostile battlefield environments. More importantly, these vehicles were air transportable, air-droppable, and swimmable, allowing planners to incorporate APCs in a much wider range of combat situations, including many "rapid deployment" scenarios. The M113s were so successful that they were quickly identified as the foundation for a family of vehicles. Early derivatives included both command post (M577) and mortar carrier (M106) configurations. Over the years, the M113 FOV has undergone numerous upgrades. In 1964, the M113A1 package replaced the original gasoline engine with a 212 horsepower diesel package, significantly improving survivability by eliminating the possibility of catastrophic loss from fuel tank explosions. Several new derivatives were produced, some based on the armoured M113 chassis (e.g., the M125A1 mortar carrier and M741 "Vulcan" air defence vehicle) and some based on the unarmoured version of the chassis (e.g., the M548 cargo carrier, M667 "Lance" missile carrier, and M730 "Chaparral" missile carrier). In 1979, the A2 package of suspension and cooling enhancements was introduced. Today's M113 fleet includes a mix of these A2 variants, together with other derivatives equipped with the most recent A3 RISE (Reliability Improvements for Selected Equipment) package. The standard RISE package includes an upgraded propulsion system (turbocharged engine and new transmission), greatly improved driver controls (new power brakes and conventional steering controls), external fuel tanks, and 200-amp alternator with four batteries. Additional A3 improvements include incorporation of spall liners and provisions for mounting external armour. The future M113A3 fleet will include a number of vehicles that will have high speed digital networks and data transfer systems. The M113A3 digitisation program includes applying hardware, software, and installation kits and hosting them in the M113 FOV. Current variants: Mechanised Smoke Obscurant System M548A1/A3 Cargo Carrier M577A2/A3 Command Post Carrier M901A1 Improved TOW Vehicle M981 Fire Support Team Vehicle M1059/A3 Smoke Generator Carrier M1064/A3 Mortar Carrier M1068/A3 Standard Integrated Command Post System Carrier OPFOR Surrogate Vehicle (OSV) Manufacturer Anniston Army Depot (Anniston, AL) United Defense, L.P. (Anniston, AL)

In school, this explicit teaching of facts and procedures is rampant in almost every subject area. In science, it’s called “the scientific method” and often includes steps such as: Observe something and/or do research. Construct a hypothesis. Make a prediction based on your hypothesis. Test your hypothesis by doing an experiment. Analyze the results of your experiment. Determine if your hypothesis was correct. The steps vary slightly between models. However, no matter the actual words on the checklist, or how many steps are included, we teach them to children as if they descended on stone tablets. Teachers devise songs or mnemonic devices to help students memorize the rigid steps. Then students memorize the vocabulary words that go along with the scientific method: hypothesis, fair test, variables, control groups, reliability, validity, etc. Finally, students fill out worksheets to match the vocabulary words with the correct definitions and put the steps in order. This is not science. Science is about wonder and risk and imagination, not checklists or vocabulary memorization. Alan Kay laments that much of what schools teach isn’t science at all, it’s science appreciation. (Kay, 2007)

Traditional managers will often object to hiring engineers with generalist skill sets, arguing that they are more expensive and that ‘I can hire two server administrators for every multi-skilled operations engineer.’” However, the business benefits of enabling faster flow are overwhelming. Furthermore, as Prugh notes, “[ I] nvesting in cross training is the right thing for [employees’] career growth, and makes everyone’s work more fun.

We will actively manage this technical debt by ensuring that we invest at least 20% of all Development and Operations cycles on refactoring, investing in automation work and architecture and non-functional requirements (NFRs, sometimes referred to as the “ilities”), such as maintainability, manageability, scalability, reliability, testability, deployability, and security. Figure 11: Invest 20% of cycles on those that create positive, user-invisible value (Source: “Machine Learning and Technical Debt with D. Sculley,” Software Engineering Daily podcast, November 17, 2015,

The deal [between product owners and] engineering goes like this: Product management takes 20% of the team’s capacity right off the top and gives this to engineering to spend as they see fit. They might use it to rewrite, re-architect, or re-factor problematic parts of the code base...whatever they believe is necessary to avoid ever having to come to the team and say, ‘we need to stop and rewrite [all our code].’ If you’re in really bad shape today, you might need to make this 30% or even more of the resources. However, I get nervous when I find teams

To achieve market orientation, we won’t do a large, top-down reorganization, which often creates large amounts of disruption, fear, and paralysis. Instead, we will embed the functional engineers and skills (e.g., Ops, QA, Infosec) into each service team, or provide their capabilities to teams through automated self-service platforms that provide production-like environments, initiate automated tests, or perform deployments.

Dan Milstein, one of the principal engineers at Hubspot, writes that he begins all blameless post-mortem meetings by saying, “We’re trying to prepare for a future where we’re as stupid as we are today.” In other words, it is not acceptable to have a countermeasure to merely “be more careful” or “be less stupid”— instead, we must design real countermeasures to prevent these errors from happening again.

Software engineering has this in common with having children: the labor before the birth is painful and difficult, but the labor after the birth is where you actually spend most of your effort. Yet

DevOps and its resulting technical, architectural, and cultural practices represent a convergence of many philosophical and management movements (including): Lean, Theory of Constraints, Toyota production system, resilience engineering, learning organizations, safety culture, Human factors, high-trust management cultures, servant leadership, organizational change management, and Agile methods.

Blameless culture originated in the healthcare and avionics industries where mistakes can be fatal. These industries nurture an environment where every “mistake” is seen as an opportunity to strengthen the system. When postmortems shift from allocating blame to investigating the systematic reasons why an individual or team had incomplete or incorrect information, effective prevention plans can be put in place. You can’t “fix” people, but you can fix systems and processes to better support people making the right choices when designing and maintaining complex systems.

A key principle of any effective software engineering, not only reliability-oriented engineering, simplicity is a quality that, once lost, can be extraordinarily difficult to recapture.

As Gordon’s book The Rise and Fall of American Growth went to press in early 2016 (its publication facilitated by digital technologies), the internet continued to disrupt countless industries while the media fanned fears of an impending ‘second machine age’, in which robots replace human workers. Gordon’s Northwestern colleague Joel Mokyr suggested that a ‘shortfall of imagination [is] largely responsible for much of today’s pessimism’. Mokyr listed a number of revolutionary new technologies then under development, including 3D printing, graphene and genetic engineering, to which might be added autonomous cars and clean energy.19 Finance writer William Bernstein accused secular stagnationists of conflating what they couldn’t conceive with that which was not possible.20 Hansen made the same mistake. The most reliable prediction, Bernstein concluded, is to assume that past economic trends continue.

Prefer experimentation over analysis. It’s far more reliable to get good at cheap validation than it is to get great at consistently picking the right solution. Even if you’re brilliant, you are almost always missing essential information when you begin designing. Analysis can often uncover missing information, but it depends on knowing where to look, whereas experimentation allows you to find problems that you didn’t anticipate.

Labgo India is a leading manufacturer of scientific instruments, providing high-quality, precision-engineered solutions for laboratories and research facilities. Specializing in cutting-edge technology, Labgo India offers a wide range of products including analytical instruments, laboratory equipment, and testing instruments, ensuring reliable performance and accuracy for scientific advancements. With a commitment to innovation and customer satisfaction, Labgo India is your trusted partner for all scientific instrumentation needs.

Which models are the most reliable? Well, obviously, the models that come from hard science and engineering are the most reliable models on this earth. And engineering quality control—at least the guts of it that matters to you and me and people who are not professional engineers—is very much based on the elementary mathematics of Fermat and Pascal: It costs so much, and you get so much less likelihood of it breaking if you spend this much. It’s all elementary high school mathematics.

As every system administrator knows, it’s one thing to know in theory that you can recover from tape, and another to know in practice exactly how to do it and how much it will cost when it becomes necessary. Practice and expertise are great drivers of efficiency and reliability.

My late mother knew absolutely nothing about cars, but had an eagle eye for people.* It would have been interesting to set her the task of buying ten cars based on her instincts about the people selling them, while at the same time tasking ten automotive engineers with acquiring ten cars at auction. I’m confident the cars my mother bought would have been every bit as reliable as the cars chosen by the engineers, perhaps more so.

engage in development tasks, because the service basically runs and repairs itself: we want systems that are automatic, not just automated. In practice, scale and new features keep SREs on their toes.

In general, an SRE team is responsible for the availability, latency, performance, efficiency, change management, monitoring, emergency response, and capacity planning of their service(s). We

Google’s Site Reliability Engineering (SRE) team has a motto: “Hope is not a strategy.

Hope is not a strategy. Traditional SRE saying

By design, it is crucial that SRE teams are focused on engineering. Without constant engineering, operations load increases and teams will need more people just to keep pace with the workload.