Jensen Nvidia Quotes

employees more space for more critical, high-level work. Employees will have a virtual assistant, almost like a brilliant intern with near-perfect memory, capable of instantly recalling any piece of knowledge stored on computers and the internet. Instead of simple file retrieval, the models can generate smarter insights drawn from the entire pool of a company’s internal data.

Do your job. Don’t be too proud of the past. Focus on the future.

People with very high expectations have very low resilience. Unfortunately, resilience matters in success,” he later said. “Greatness is not intelligence. Greatness comes from character.”17 And character, in his view, can only be the result of overcoming setbacks and adversity. To Jensen, the struggle to persevere in the face of bad, and often overwhelming, odds is simply what work is. It is why, whenever someone asks him for advice on how to achieve success, his answer has been consistent over the years: “I wish upon you ample doses of pain and suffering.

I don’t actually know anybody who is incredibly successful who just approaches business like, ‘This is just business. This is what I do from 8 to 5, and I’m going home, and at 5:01, I’m shutting it down,’ ” Jensen has said.15 “I’ve never known anybody who is incredibly successful like that. You have to allow yourself to be obsessed with your work.

The Transformer was a big deal,” Jensen said in 2023. “The ability for you to learn patterns and relationships from spatial as well as sequential data must be an architecture that’s very effective, right? And so I think on its first principles, you can kind of think Transformer’s going to be a big, big deal. Not only that, you could train it in parallel and you can really scale this model up.”7

Jensen’s at-times harsh approach was a deliberate choice. He knew that people would inevitably fail, especially in a high-pressure industry. He wanted to offer employees more opportunities to prove themselves, believing that they, in every case, are often just one or two epiphanies away from solving their problems themselves. “I don’t like giving up on people,” he said. “I’d rather torture them into greatness.

The truth, not widely understood until later, was that the deep-learning revolution was as much a revolution in hardware as software. It was the product of not one but two unpopular, cast-off, discredited, and cash-starved technologies whose ideal form could only be revealed in synthesis. Neural nets running on parallel computers: these tightly coupled technologies were the twin strands of DNA for a new and powerful organism, looking to consume all the data in the world.

According to Jeff Fisher, Nvidia’s approach involves “no magic.” It’s just hard work and ruthless efficiency, all in the service of maintaining competitive advantage. And everyone who works with Nvidia must embrace it, not just its internal teams.4 Everything the tiger teams did was expensive and resulted in a drag on the bottom line. Yet Nvidia has always been willing to use its financial resources to invest in critical parts of the business—even when that has meant other companies’ business.

Structural prediction and protein design, once considered impossible problems, are now solvable. Grigoryan explains that the complexity of a protein and its possible states surpasses the number of atoms in the universe. “Those numbers are extremely challenging for any computational tools to deal with,” he said. But he believes a skilled protein biophysicist can examine a particular molecular structure and deduce its potential functions, suggesting there may be learnable general principles in nature—exactly the sort of operation that a “universal prediction engine” such as AI should be able to figure out. Generate:Biomedicines has applied AI to examine and map molecules at the cell level, and Grigoryan sees the potential to extend the same technique to the entire human body. Simulating how the human body will react is orders of magnitude more complicated, but Grigoryan thinks it will be possible. “Once you see it working, it’s hard to imagine it doesn’t just continue,” he said, referring to the power of AI.

Nvidia was founded in 1993 by Chris Malachowsky, Curtis Priem, and Jensen Huang, the latter of whom remains CEO today.

But as Garlick acclimated to these shortcuts, he came to see the value of the Nvidia approach. “There was a bizarre brilliance to it all: just iterate, iterate, iterate, execute, execute, execute,” he said. “The way I see it now, tech debt is the battle scar of the survivor.

Je vášnivým kuchařem, rád vaří pro své hosty. A také cyklistou, často se účastní charitativních cyklistických akcí. Je milovníkem klasické hudby a jazzu, rád navštěvuje živá představení a podporuje místní umělce. Zajímá se také o výtvarné umění a divadlo. Je náruživým čtenářem s širokým záběrem zájmů od technologií a sci-fi po historii a filozofii. Čtení považuje za důležitý zdroj inspirace a neustálého vzdělávání. Tím se Jensen Huang vymyká z tradičního kultu „tech bro’s“, který je tak rozšířený mezi šéfy technologických firem v Silicon Valley. Žádné extrémní hobby, které by ho odváděly od hlavního zaměření firmy, jakým holduje Jeff Bezos, Mark Zuckerberg nebo Elon Musk. Žádné sexuální skandály, jako Trevis Kalanick, někdejší šéf Uberu nebo třeba Andy Rubin, zakladatel Androidu, kterého za to Google zbavil vedení Androidu. Žádné milenky, zástupy nemanželských dětí, nadřazené chování. Naopak je Jensen Huang vnímán až jako asketicky žijící člověk - pokud se to dá říct o člověku, který na veřejnosti vystupuje v kožené bundě v ceně kolem 100 000 Kč a jehož majetek se odhaduje na 100 miliard dolarů (stále drží kolem 4 % akcií Nvidia).

Under Jensen Huang, Nvidia didn't just ride the AI wave; they built the surfboard, the wave pool, and then taught everyone how to surf through strategic bets on GPUs and CUDA

The bias against neural nets, Hinton felt, was “ideological,” a word he pronounced in the same venomous tone that Huang had used to say “political.” The ideology of the research community at the time was that it was not enough that AI be useful. Instead, AI should somehow “unlock” the secrets of intelligence and encode them in math. The standard, 1,100-page AI textbook of the time was a survey of probabilistic reasoning, decision trees, and support-vector machines. The neural nets got just ten pages, with a brief discussion of backgammon up front. When Hinton’s colleague designed a neural net that outperformed state-of-the-art software for recognizing pedestrians, he couldn’t even get his paper admitted to a conference. “The reaction was well, that doesn’t count, because it doesn’t explain how the computation is done—it’s just not telling us anything,” Hinton said.

Catanzaro was uncorked now—I sensed that he didn’t often get to share this perspective at his job. “It doesn’t need to inhabit this biosphere. In fact, it doesn’t need to be on the Earth, either, because the thing about artificial intelligence is that it travels at the speed of light. Humans, you know, we actually have to lug bodies around. Artificial intelligence can move along a radio signal as long as there’s an antenna on the other side.” Free of the limitations of biology, Catanzaro explained, AI would rapidly spread throughout the solar system and beyond. “Humans are naturally confrontational—like, we’re territorial animals, and it’s built into our limbic system to defend our turf,” he said. “AI, if it’s truly intelligent, the things that it’s interested in are so much bigger than the little thin crust of Earth that the humans live on. I don’t think that it’s going to be interested in taking that from us. Rather, I feel like AI is going to want to take care of us.” • • • Serving as a zoo animal for a space computer was an experience to be savored at leisure.

The executives were more afraid of Jensen yelling at them than they were of wiping out the human race.

And that’s not even close to the most expensive Nvidia product. Nvidia’s latest server rack system as of this writing, the Blackwell GB200 series, was specifically designed to train “trillion-parameter” AI models. It comes with seventy-two GPUs and costs $2 million to $3 million—the most expensive Nvidia machine ever made. The company’s top-end-product pricing isn’t merely increasing; it is accelerating.

Current AI models can now understand requests via context and because they can grasp natural conversational language. It is a major breakthrough. “The core of generative AI is the ability for software to understand the meaning of data,” Jensen said.16 He believes that companies will “vectorize” their databases, indexing and capturing representations of information and connecting it to a large language model, enabling users to “talk to their data.

Nvidia remains the only stand-alone graphics-chip firm to this day, even though hundreds of others have thrown their hats in the ring. Jensen himself is now the technology industry’s longest-serving CEO.

Even though they had not yet made the move to the United States themselves, his parents wanted to send their children to an American boarding school so they could get a good education. They found one called Oneida Baptist Institute, which was located in eastern Kentucky and accepted international students. They could afford the tuition only by selling nearly all of their possessions. Jensen remembers the initial drive through the mountains of Kentucky, past the single building that was the town of Oneida’s only gas station, grocery store, and post office all at once. The boarding school had around three hundred students, evenly split between boys and girls. But it was not a prep school as Jensen’s family originally thought. Oneida Baptist Institute was, instead, a reform school for troubled young people. It had been founded in the 1890s to remove children from feuding families in the state and thus keep them from killing each other.

Most of the other possibilities on Priem’s list incorporated “NV” as a reference to their first planned chip design. These names included iNVention, eNVironment, and iNVision—the kinds of everyday words that other companies had already co-opted for their own brands, such as a toilet paper company that had trademarked the name “Envision” for its environmentally sustainable product line. Another name was too similar to the brand of a computer-controlled toilet. “These names were all stinky,” Priem said.

The last remaining option was “Invidia,” which Priem found by looking up the Latin word for envy—in a sense, another callback to their work on the GX, when he and Malachowsky believed that their rivals, both within and beyond Sun, had envied their success. “We dropped the ‘I’ and went with NVidia to honor the NV1 chip we were developing,” said Priem, “and secretly hoped that someday Nvidia would be something that would be envied.” With a name in hand, Jensen sought out a lawyer and chose James Gaither, who worked at the law firm of Cooley Godward. Gaither’s firm was midsized, with fewer than fifty attorneys on staff. Even so, it had carved out a niche for itself as the go-to firm for early-stage Silicon Valley start-ups. During their first meeting, Gaither asked Jensen how much money he had in his pocket. Jensen said $200. “Hand it over,” said Gaither. He then told Jensen he now owned a large equity stake in Nvidia.

Nobody goes to the store to buy a Swiss Army knife. It’s something you get for Christmas.

So Jensen asked employees at every level of the organization to send an e-mail to their immediate team and to executives that detailed the top five things they were working on and what they had recently observed in their markets, including customer pain points, competitor activities, technology developments, and the potential for project delays. “The ideal top five e-mail is five bullet points where the first word is an action word. It has to be something like finalize, build, or secure,” said early employee Robert Csongor.

The “Top 5” e-mails became a crucial feedback channel for Jensen. They enabled him to get ahead of changes in the market that were obvious to junior employees but not yet to him or his e-staff. “I’m looking to detect the weak signals,” he would tell his employees when asked why he liked the Top 5 process. “It’s easy to pick up the strong signals, but I want to intercept them when they are weak.” To his e-staff, he was a little more pointed. “Don’t take this the wrong way, but you may not have the brainpower or the wherewithal to detect something I think is pretty significant.”15

Ernst felt Jensen was growing frustrated and would soon leave the table, so he decided to ask him something different. “Jensen, I have a two-year-old daughter at home. I bought a new Sony A100 DSLR camera and regularly download photos to my Mac to do some light editing in Photoshop. But whenever I do this, my Mac slows down as soon as I open one of these high-resolution images. It’s even worse on my Think-Pad. Can a GPU solve this problem?” Jensen’s eyes lit up. “Don’t write about this because it’s not out yet, but Adobe is a partner of ours. Adobe Photoshop with CUDA can instruct the CPU to off-load the task to the GPU, and make it much faster,” he said. “That’s exactly what I’m talking about with the coming ‘Era of the GPU.

Before making the transcontinental move, however, Malachowsky decided to apply for jobs at other companies, solely for the purpose of getting some practice interviewing. His first invitation came from the nascent supercomputer division at Evans and Sutherland, a graphics company otherwise known for making high-end flight simulators for military training. He was rejected right away; his interviewers thought he questioned the status quo too much and felt that he would be a poor fit at the company. (Malachowsky believed their feedback didn’t bode well for the company’s future. He was right. Evans and Sutherland’s first supercomputer later failed to sell, and the looming end of the Cold War meant that simulator demand from the military was already drying up.)

We were diluted across too many different areas,” Jensen recalled.8 “We learned it was better to do fewer things well than to do too many things even though it looked good on a PowerPoint slide. Nobody goes to the store to buy a Swiss Army knife. It’s something you get for Christmas.”9

The more you buy, the more you save.

Jensen himself has called AI a “universal function approximator” that can predict the future with reasonable accuracy. This applies as much in “high-tech” fields such as computer vision, speech recognition, and recommendations systems as it does in “low-tech” tasks such as correcting grammar or analyzing financial data. He believes that eventually it will apply to “almost anything that has structure.

When those former 3dfx engineers arrived at Nvidia, they expected to find out that their victorious rival had some kind of unique process or technology that allowed them to make new chips every six months. Dwight Diercks remembers their shock when they found out that the explanation was much simpler. “Oh my God, we got here and we thought there was going to be a secret sauce,” one engineer said.3 “It turns out it’s just really hard work and intense execution on schedules.” It was Nvidia’s culture, in other words, that made the difference.

While some Nvidia employees were initially worried about the lawsuit, Andrew Logan, Nvidia’s director of corporate marketing, was excited. “I got a call from the Wall Street Journal right now on my voicemail,” he told his colleagues after the lawsuit was announced. “This is perfect. We’re on the map!

Within just a few years, Nvidia’s success in graphics made it one of TSMC’s top two or three customers. Tsai remembered Jensen would negotiate hard over pricing and would repeatedly cite how the graphics company had only a 38 percent gross margin. One particular dispute prompted Tsai to travel to California and meet with Jensen at a restaurant that wasn’t much better than Denny’s. “We tried to resolve the dispute. I forgot the details,” Tsai said. “But it really hit me. Jensen taught me his philosophy of doing business called ‘rough justice.’ ” Jensen explained that “rough” meant the relationship was not flat but rather had ups and downs. Justice was the important part. “After a certain period of time, let’s say a few years, it would net out to roughly equal.” To Tsai, this was a way of describing a win-win partnership, though one that acknowledged there wouldn’t be a win-win every single time. Sometimes one side would get the better of a specific deal or incident, and the next time it would be the other side. As long as it was roughly 50-50 after a few years—not 60-40 or 40-60—it was a positive relationship. He remembers thinking Jensen’s approach made a great deal of sense.

Chris Malachowsky came to the rescue once more. “Why don’t we test every chip and run software on every part?” he asked one day. “You can’t possibly do that,” another Nvidia executive replied. “Why?” asked Malachowsky.15

Priem’s design had a software-based “resource manager,” essentially a miniature operating system that sat on top of the hardware itself. The resource manager allowed Nvidia’s engineers to emulate certain hardware features that normally needed to be physically printed onto chip circuits. This involved a performance cost but accelerated the pace of innovation, because Nvidia’s engineers could take more risks. If the new feature wasn’t ready to work in the hardware, Nvidia could emulate it in software. At the same time, the engineers could take hardware features out when there was enough leftover computing power, saving chip die area.

Nvidia’s rapid iteration meant that “the competition will always be shooting behind the duck,” as Jensen described it. Like a hunter who aims at a moving target instead of ahead of it, other graphics-chip makers would lag behind—there would be too many new chips coming out too fast. Nvidia’s competitors would simply be overwhelmed. “The number one feature of any product is the schedule,” Jensen later said.

Notably, all eight Google scientists who authored the seminal “Attention Is All You Need” paper on the Transformer deep-learning architecture—which proved foundational for advancements in modern AI large language models (LLMs), including the launch of ChatGPT—soon after left Google to pursue AI entrepreneurship elsewhere. “It’s just a side effect of being a big company,” said Llion Jones, one of the coauthors of the Transformer paper.4 “I think the bureaucracy [at Google] had built to the point where I just felt like I couldn’t get anything done,” he added, expressing frustration with his inability to access resources and data.

Keane was also surprised by the sheer openness he found at Nvidia. He joined at the general-manager level and was allowed to attend every board meeting and off-site board event. When a typical CEO would have eight or nine people in a room for big executive meetings, Jensen would have a packed house. “Everyone could hear what he was telling the executive staff,” Keane said. “It kept everybody in sync.” When there is important information to share or an impending change in the direction of the business, Jensen says he tells everybody at Nvidia at the same time and asks for feedback. “It turns out that by having a lot of direct reports, not having one-on-ones, [we] made the company flat, information travels quickly, employees are empowered,” Jensen said. “That algorithm was well conceived.

Nvidia doesn’t constantly fire people and rehire them,” said Jay Puri, head of global field operations.12 “We take people that we have and we are able to redirect them into a new mission.” Managers at Nvidia were trained not to get territorial or feel like they “own” their people and instead got used to them moving around between task groups. This practice removed one of the main sources of friction at large companies. “Managers don’t feel like they get power by having large teams,” Puri continued. “You get power at Nvidia by doing amazing work.

We thought we had built great technology and a great product,” Malachowsky said. “It turns out we only built great technology. It wasn’t a great product.

DURING ONE OF NVIDIA’S VERY FIRST board meetings, director Harvey Jones, a former CEO of a leading chip-design-software company called Synopsys, asked Jensen about the NV1: “How would you position this?” At the time, Jensen didn’t realize that Jones was not merely asking about the NV1’s feature set or product specifications. He was asking him to consider how Nvidia would sell the new chip in a highly competitive industry. He knew that products had to be presented in the clearest, most precise terms in order to stand out. “He asked me a simple question. I had no idea how simple it was. It was impossible for me to answer because I didn’t understand it,” Jensen remembered.11 “The answer is supremely deep. You’ll spend your whole career answering that question.

Vivoli knew that graphics demos would have the most impact if Nvidia had a thorough understanding of its audience. Earlier demos were targeted at engineers, which is why they showed off the specific features and capabilities of Nvidia’s new chips. Like the 3-D cube that Voodoo Graphics impressed the Hambrecht & Quest conference with in 1996, such demos were only impressive if you knew the computations that were occurring “under the hood.” A nonengineer wouldn’t necessarily know what he or she was looking at. So Vivoli changed the focus of demos away from a cold demonstration of graphics performance—the visual equivalent of reading a list of benchmark metrics—and gave them a sentimental side.

The hardware engineer replied, “We don’t think it’s a big deal. It’s kind of untested.” The developer-relations employee was aghast. “What are you talking about? ATI is shipping this feature in a product already, and gamers love it.” Once again, Nvidia’s own engineers seemed to be unaware of what the market wanted. “NV30 was an architectural disaster. It was an architectural tragedy,” Jensen later said.11 “The software team, the architecture team, and the chip-design team hardly communicated with each other.

It was imperative that both groups work in close harmony. “Any computer architecture has a software side and a hardware side. CUDA is not just a piece of software,” said Andy Keane, a former general manager for Nvidia’s data-center business. “It’s a representation of the machine. It’s a way you access the machine, so they have to be designed together.”5

Now I saw where the fear was coming from. The executives were more afraid of Jensen yelling at them than they were of wiping out the human race.

But the final word went to Morris Chang. He didn’t attribute Huang’s success to his work ethic, which, at TSMC, would have been considered slightly above average—nor did he find him especially adaptable. Chang was ninety-two years old when I spoke with him, wearing a purple corded sweater and sitting in front of a striking piece of abstract art, his face serene, his hair completely white. In seventy years of corporate life across two continents he had seen every manner of executive there was to see. To him, the explanation was simple, and there was no secret: “His intellect is just superior.

For Dally, it was Huang’s tireless work ethic that made Nvidia succeed. Even Dally, who left no spare second in his day, could not quite believe the superhuman efforts of his boss. “The rest of us are just here to reduce the bandwidth demands on Jensen,” Dally said. “I mean, when does he sleep?” Diercks agreed: “His hobbies are work, email, and work.

I FOUND THIS TO BE A pervasive attitude within Nvidia: that the culture of the place discourages looking back, whether at errors or successes, in favor of focusing on the future—the blank whiteboard of opportunity.

Krizhevsky himself escaped the dragnet. Rarely saying a word to anyone, he was not the ideal collaborator, and he departed Google in 2017. His share of the auction money was just under $15 million, enough not to work anymore, especially given the asceticism of his lifestyle. In 2019 he granted a Japanese news crew a visit inside his Bay Area home. Krizhevsky lived like a Benedictine monk, in a spartan apartment above a Vietnamese restaurant. The walls inside were completely bare; the only items of furniture were a desk, a couch, a digital piano, and a television; the only sign of life in the place was his house cat. Krizhevsky, the Orville Wright of the neural net, told the news crew he had walked away from the technology. “Maybe it’s just my personality,” he said, “but when I spend a very long time specializing in something, after about ten years I start to lose interest.

AlexNet had used 650,000 neurons to represent 630,000,000 synaptic connections. At that scale, any one synapse hardly mattered. Serial code was so finicky that sometimes a single misplaced semicolon could crash an entire operating system, but for neural nets a bad weight was just one data point among millions. For this reason, as they developed cuDNN, Nvidia programmers rebalanced the trade-off between precision and speed. Good neural-net software, they reasoned, should favor the latter.

When Google’s offer reached $44 million, Hinton, with the endorsement of Sutskever and Krizhevsky, cut the auction off and took the money. Google, the three felt, was a more natural cultural fit than Baidu. AlexNet, the neural network that Krizhevsky trained in his bedroom, could now be mentioned alongside the Wright Flyer and the Edison bulb. “That was a kind of Big Bang moment,” Hinton said. “That was the paradigm shift.

The conviction that Huang had been given a once-in-a-lifetime opportunity seized him. The acronym “O.I.A.L.O.” was repeated at every meeting. From the day Huang had started his career, at twenty, he had worked relentlessly, putting in consecutive twelve-hour days, six days a week, for three decades straight. Now past fifty, and with his kids grown, he began to work even harder.

Huang encouraged his employees to preserve the mindset they’d adopted during the Riva crunch, asking them to constantly behave as if the company was teetering on the verge of bankruptcy even when it was making massive profits. For years to come, Jensen opened staff presentations with the words “Our company is thirty days from going out of business.” Even today at Nvidia, this sentence remains the corporate mantra.

The average CEO will try to listen to the customer, but in computing, that’s a big mistake, because customers just don’t know what’s possible. They just don’t know what can be done!

The guy had an MBA, but he’d never read a book about pricing,” Diercks said. “Jensen had read probably ten or fifteen.” As the argument progressed, Huang halted the discussion and asked the MBA to name his three favorite books on pricing. The guy fumbled around for a bit, unable to name a single title. Huang listed out his three favorites, then told the executive he’d resume the discussion once he’d finished them.

Huang’s best-loved business book was The Innovator’s Dilemma, by the Harvard Business School professor Clayton Christensen. First published in 1997, the book popularized the term disruptive innovation to describe how incumbent firms lose out to start-up competitors. Although the word disruption has since grown meaningless through overuse, the source material is worth revisiting. In Christensen’s model, small firms can chisel away at large ones by serving niche, marginally profitable customers that the market leaders have dismissed.

There are times when it is right not to listen to customers, right to invest in lower-performing products that result in lower margins, and right to pursue small, rather than substantial, markets.” It was a point that the buzzword discussion of “disruption” in the popular press usually missed.

But in ceding the PC gaming market to Nvidia, the workstation companies had made a fatal mistake, just as GM had by ignoring Honda decades earlier. Nvidia, like Honda, was today selling low-margin products to teenage boys, but if the analogy held, tomorrow they might overtake the business workstations of Sun Microsystems and SGI. Sometimes Jensen would even speak to his clustered executive staff about the possibility of disrupting Intel, then one of the most valuable firms on Earth. In the meantime, Nvidia would survive in Intel’s territory through a strategy of continuous tactical retreat. “To this day, we don’t compete with Intel,” Huang said in 2023, describing their Tom-and-Jerry relationship. “Whenever they come near us, I pick up my chips and run.” The gospel of Christensen counseled Nvidia to sell offbeat products that Intel would not conceive of making to customers it would never want to serve. “Jensen would tell us, even back then, that Nvidia could someday be bigger than Intel,” Kirk recalled. “It was just a question of strategy.

Kirk recalled showing off the Riva 128 to a competitor at a trade show. When the engineer saw what it could do, he gave up on the spot. “I hired him within a few days—and that killed that company, right?” Kirk said. “Because, you know, I removed its brain.” Kirk, mild and professorial, had a predator’s instincts. “We had all the geniuses from all the other start-ups, and as we were successful in overtaking more and more of these little companies, the remaining companies had a harder and harder time staying alive.

Huang educated himself by reading every business book he could find. “If you go into his office today, it’s totally abandoned; he never uses it,” one employee told me. “But it’s filled with stacks and stacks of business books.

Hans Mosesmann, a veteran industry analyst who would later help manage Nvidia’s IPO, recalled talking with one of Huang’s former managers at LSI who’d been tasked with giving Huang an employee evaluation. The evaluation form resembled a report card, but the manager left the grades blank. At the bottom, the manager wrote, “Jensen is an excellent employee. I look forward to working for him some day.

Decision-making, for Huang, was a clinical process with little room for useless emotions like hope. To him, business was just another engineering problem. Engineers looked to break down complex problems into simple governing principles, which could then be leveraged to powerful effect. To launch a start-up, then, Huang needed first to understand those principles. He had to research the market, the supply chain, the competition, the technology, and the product fit. He had to arrange exploratory phone calls with customers, game developers, and computer-graphics experts. He had to pore through years of dull state-of-the-industry reports, paging through bar charts and sales figures and customer surveys in search of a glimmer of an upward trend.

Nvidia had accrued a great deal of “tech debt,” repeatedly taking shortcuts that led over time to less maintainable code and creating problems for programmers later on. But as Garlick acclimated to these shortcuts, he came to see the value of the Nvidia approach. “There was a bizarre brilliance to it all: just iterate, iterate, iterate, execute, execute, execute,” he said. “The way I see it now, tech debt is the battle scar of the survivor.

The rumors about Huang by this point were almost mythologically sinister: he poached employees; he lifted ideas; he spun the reviewers; he kicked his fallen competitors in the teeth. Mostly, though, they hated Huang because he had whipped their collective corporate ass. “Nvidia made enemies all along the way as they rose to power, including with partners and suppliers,” Peddie said. “Jensen, you could say he’s a personal friend of mine—but he was ruthless.

Gaming companies rarely had a dress code, and some coders indulged in performative slobbishness. Karen Huaulme, another 3dfx hire, recalled getting lost in a suburban Dallas office complex looking for the headquarters of iD Software, the maker of Quake. She resolved her problem by following the worst-dressed person she could find, a pasty young man with long stringy hair in flip-flops and a tattered T-shirt. He led her straight to iD’s front door.

Huang was now a centimillionaire, but his newfound wealth did not distract from his objective of crushing and absorbing the competition until only his firm remained. Dwight Diercks recalled no parties, no champagne, no sense of relief, not even congratulations from the boss. He shared with me an email he had saved from Huang. “The TNT2 team needs to do whatever it takes to get over the finish line,” Huang had written. The email continued in a tone of panicked desperation, with Huang complaining about missed deadlines and fretting over ascendant competitors. Huang’s life would be transformed by the influx of wealth from Nvidia’s new shareholders, but he evinced no sense of victory or pride—none whatsoever. “Get it done,” Huang ended tersely. Diercks shook his head in amazement. “He wrote that the day after the IPO,” he said.

Once you understand the physical limits of what is possible, you understand the competition can’t go any faster either.

The reason that Nvidia succeeded was not that its circuits were better. The reason was that its software was better.

Between 2012 and 2022, Nvidia achieved a thousand-fold speed-up in single-chip AI inference performance, which was far in excess of anything that Moore’s Law had ever achieved. A mere 2.5× of that speed-up came from transistors; most of the remaining 400× came from Nvidia’s mathematical toolbox. “Honestly, AMD can make silicon just as good as we can,” Arjun Prabhu said. “They just can’t make the calculations go as fast.

The ability of GPT-2 to answer new questions without explicit training was an example of the “emergent” properties in AI. These unexpected skills and behaviors were appearing as the models grew larger, surprising even the researchers. Once a model crossed the threshold of emergence, no one, not even its designers, could say what it was fully capable of doing.

In early 2024, an administrator at CalTech’s data center told me that the school’s wait time for delivery on an H100 chip was almost eighteen months. He had encouraged professors at the school to switch to other providers but found few willing to accept. “They’d rather wait for the hardware than switch away from CUDA,” he said. It was all this code that made Nvidia hard to compete against. Upstarts might design a new chip, but that wasn’t enough—Dwight Diercks, Nvidia’s head of software engineering, had ten thousand programmers working for him. “We’re really a software company; that’s the thing people don’t understand,” Diercks said.

Plenty of people worked long hours, though. Jens Horstmann attributed Huang’s success to his adaptability. “I’ve often asked myself, how is it that we started in the same cubicle, you know, with a similar IQ, both working equally hard,” Horstmann said. “How is it that this person not only built this amazing company, but also a network around him of people that—that would just die for him if needed?” Huang, Horstmann believed, had changed himself many times. He recalled Huang at LSI, pushing the simulation software to its outer limits. “Now, he’s still doing the same thing, but what he’s engineering is himself. He was not born as a great CEO; he was not destined to be one. He transformed himself into one, just by abstracting!