Jensen Huang Ai Quotes

The truth, not widely understood until later, was that the deep-learning revolution was as much a revolution in hardware as software. It was the product of not one but two unpopular, cast-off, discredited, and cash-starved technologies whose ideal form could only be revealed in synthesis. Neural nets running on parallel computers: these tightly coupled technologies were the twin strands of DNA for a new and powerful organism, looking to consume all the data in the world.

Structural prediction and protein design, once considered impossible problems, are now solvable. Grigoryan explains that the complexity of a protein and its possible states surpasses the number of atoms in the universe. “Those numbers are extremely challenging for any computational tools to deal with,” he said. But he believes a skilled protein biophysicist can examine a particular molecular structure and deduce its potential functions, suggesting there may be learnable general principles in nature—exactly the sort of operation that a “universal prediction engine” such as AI should be able to figure out. Generate:Biomedicines has applied AI to examine and map molecules at the cell level, and Grigoryan sees the potential to extend the same technique to the entire human body. Simulating how the human body will react is orders of magnitude more complicated, but Grigoryan thinks it will be possible. “Once you see it working, it’s hard to imagine it doesn’t just continue,” he said, referring to the power of AI.

Under Jensen Huang, Nvidia didn't just ride the AI wave; they built the surfboard, the wave pool, and then taught everyone how to surf through strategic bets on GPUs and CUDA

The bias against neural nets, Hinton felt, was “ideological,” a word he pronounced in the same venomous tone that Huang had used to say “political.” The ideology of the research community at the time was that it was not enough that AI be useful. Instead, AI should somehow “unlock” the secrets of intelligence and encode them in math. The standard, 1,100-page AI textbook of the time was a survey of probabilistic reasoning, decision trees, and support-vector machines. The neural nets got just ten pages, with a brief discussion of backgammon up front. When Hinton’s colleague designed a neural net that outperformed state-of-the-art software for recognizing pedestrians, he couldn’t even get his paper admitted to a conference. “The reaction was well, that doesn’t count, because it doesn’t explain how the computation is done—it’s just not telling us anything,” Hinton said.

Catanzaro was uncorked now—I sensed that he didn’t often get to share this perspective at his job. “It doesn’t need to inhabit this biosphere. In fact, it doesn’t need to be on the Earth, either, because the thing about artificial intelligence is that it travels at the speed of light. Humans, you know, we actually have to lug bodies around. Artificial intelligence can move along a radio signal as long as there’s an antenna on the other side.” Free of the limitations of biology, Catanzaro explained, AI would rapidly spread throughout the solar system and beyond. “Humans are naturally confrontational—like, we’re territorial animals, and it’s built into our limbic system to defend our turf,” he said. “AI, if it’s truly intelligent, the things that it’s interested in are so much bigger than the little thin crust of Earth that the humans live on. I don’t think that it’s going to be interested in taking that from us. Rather, I feel like AI is going to want to take care of us.” • • • Serving as a zoo animal for a space computer was an experience to be savored at leisure.

And that’s not even close to the most expensive Nvidia product. Nvidia’s latest server rack system as of this writing, the Blackwell GB200 series, was specifically designed to train “trillion-parameter” AI models. It comes with seventy-two GPUs and costs $2 million to $3 million—the most expensive Nvidia machine ever made. The company’s top-end-product pricing isn’t merely increasing; it is accelerating.

Current AI models can now understand requests via context and because they can grasp natural conversational language. It is a major breakthrough. “The core of generative AI is the ability for software to understand the meaning of data,” Jensen said.16 He believes that companies will “vectorize” their databases, indexing and capturing representations of information and connecting it to a large language model, enabling users to “talk to their data.

Jensen himself has called AI a “universal function approximator” that can predict the future with reasonable accuracy. This applies as much in “high-tech” fields such as computer vision, speech recognition, and recommendations systems as it does in “low-tech” tasks such as correcting grammar or analyzing financial data. He believes that eventually it will apply to “almost anything that has structure.

Notably, all eight Google scientists who authored the seminal “Attention Is All You Need” paper on the Transformer deep-learning architecture—which proved foundational for advancements in modern AI large language models (LLMs), including the launch of ChatGPT—soon after left Google to pursue AI entrepreneurship elsewhere. “It’s just a side effect of being a big company,” said Llion Jones, one of the coauthors of the Transformer paper.4 “I think the bureaucracy [at Google] had built to the point where I just felt like I couldn’t get anything done,” he added, expressing frustration with his inability to access resources and data.

Now I saw where the fear was coming from. The executives were more afraid of Jensen yelling at them than they were of wiping out the human race.

But the final word went to Morris Chang. He didn’t attribute Huang’s success to his work ethic, which, at TSMC, would have been considered slightly above average—nor did he find him especially adaptable. Chang was ninety-two years old when I spoke with him, wearing a purple corded sweater and sitting in front of a striking piece of abstract art, his face serene, his hair completely white. In seventy years of corporate life across two continents he had seen every manner of executive there was to see. To him, the explanation was simple, and there was no secret: “His intellect is just superior.

For Dally, it was Huang’s tireless work ethic that made Nvidia succeed. Even Dally, who left no spare second in his day, could not quite believe the superhuman efforts of his boss. “The rest of us are just here to reduce the bandwidth demands on Jensen,” Dally said. “I mean, when does he sleep?” Diercks agreed: “His hobbies are work, email, and work.

Krizhevsky himself escaped the dragnet. Rarely saying a word to anyone, he was not the ideal collaborator, and he departed Google in 2017. His share of the auction money was just under $15 million, enough not to work anymore, especially given the asceticism of his lifestyle. In 2019 he granted a Japanese news crew a visit inside his Bay Area home. Krizhevsky lived like a Benedictine monk, in a spartan apartment above a Vietnamese restaurant. The walls inside were completely bare; the only items of furniture were a desk, a couch, a digital piano, and a television; the only sign of life in the place was his house cat. Krizhevsky, the Orville Wright of the neural net, told the news crew he had walked away from the technology. “Maybe it’s just my personality,” he said, “but when I spend a very long time specializing in something, after about ten years I start to lose interest.

AlexNet had used 650,000 neurons to represent 630,000,000 synaptic connections. At that scale, any one synapse hardly mattered. Serial code was so finicky that sometimes a single misplaced semicolon could crash an entire operating system, but for neural nets a bad weight was just one data point among millions. For this reason, as they developed cuDNN, Nvidia programmers rebalanced the trade-off between precision and speed. Good neural-net software, they reasoned, should favor the latter.

When Google’s offer reached $44 million, Hinton, with the endorsement of Sutskever and Krizhevsky, cut the auction off and took the money. Google, the three felt, was a more natural cultural fit than Baidu. AlexNet, the neural network that Krizhevsky trained in his bedroom, could now be mentioned alongside the Wright Flyer and the Edison bulb. “That was a kind of Big Bang moment,” Hinton said. “That was the paradigm shift.

The conviction that Huang had been given a once-in-a-lifetime opportunity seized him. The acronym “O.I.A.L.O.” was repeated at every meeting. From the day Huang had started his career, at twenty, he had worked relentlessly, putting in consecutive twelve-hour days, six days a week, for three decades straight. Now past fifty, and with his kids grown, he began to work even harder.

Nvidia had accrued a great deal of “tech debt,” repeatedly taking shortcuts that led over time to less maintainable code and creating problems for programmers later on. But as Garlick acclimated to these shortcuts, he came to see the value of the Nvidia approach. “There was a bizarre brilliance to it all: just iterate, iterate, iterate, execute, execute, execute,” he said. “The way I see it now, tech debt is the battle scar of the survivor.

The rumors about Huang by this point were almost mythologically sinister: he poached employees; he lifted ideas; he spun the reviewers; he kicked his fallen competitors in the teeth. Mostly, though, they hated Huang because he had whipped their collective corporate ass. “Nvidia made enemies all along the way as they rose to power, including with partners and suppliers,” Peddie said. “Jensen, you could say he’s a personal friend of mine—but he was ruthless.

Gaming companies rarely had a dress code, and some coders indulged in performative slobbishness. Karen Huaulme, another 3dfx hire, recalled getting lost in a suburban Dallas office complex looking for the headquarters of iD Software, the maker of Quake. She resolved her problem by following the worst-dressed person she could find, a pasty young man with long stringy hair in flip-flops and a tattered T-shirt. He led her straight to iD’s front door.

Huang was now a centimillionaire, but his newfound wealth did not distract from his objective of crushing and absorbing the competition until only his firm remained. Dwight Diercks recalled no parties, no champagne, no sense of relief, not even congratulations from the boss. He shared with me an email he had saved from Huang. “The TNT2 team needs to do whatever it takes to get over the finish line,” Huang had written. The email continued in a tone of panicked desperation, with Huang complaining about missed deadlines and fretting over ascendant competitors. Huang’s life would be transformed by the influx of wealth from Nvidia’s new shareholders, but he evinced no sense of victory or pride—none whatsoever. “Get it done,” Huang ended tersely. Diercks shook his head in amazement. “He wrote that the day after the IPO,” he said.

Once you understand the physical limits of what is possible, you understand the competition can’t go any faster either.

Huang encouraged his employees to preserve the mindset they’d adopted during the Riva crunch, asking them to constantly behave as if the company was teetering on the verge of bankruptcy even when it was making massive profits. For years to come, Jensen opened staff presentations with the words “Our company is thirty days from going out of business.” Even today at Nvidia, this sentence remains the corporate mantra.

The average CEO will try to listen to the customer, but in computing, that’s a big mistake, because customers just don’t know what’s possible. They just don’t know what can be done!

The guy had an MBA, but he’d never read a book about pricing,” Diercks said. “Jensen had read probably ten or fifteen.” As the argument progressed, Huang halted the discussion and asked the MBA to name his three favorite books on pricing. The guy fumbled around for a bit, unable to name a single title. Huang listed out his three favorites, then told the executive he’d resume the discussion once he’d finished them.

Huang’s best-loved business book was The Innovator’s Dilemma, by the Harvard Business School professor Clayton Christensen. First published in 1997, the book popularized the term disruptive innovation to describe how incumbent firms lose out to start-up competitors. Although the word disruption has since grown meaningless through overuse, the source material is worth revisiting. In Christensen’s model, small firms can chisel away at large ones by serving niche, marginally profitable customers that the market leaders have dismissed.

There are times when it is right not to listen to customers, right to invest in lower-performing products that result in lower margins, and right to pursue small, rather than substantial, markets.” It was a point that the buzzword discussion of “disruption” in the popular press usually missed.

But in ceding the PC gaming market to Nvidia, the workstation companies had made a fatal mistake, just as GM had by ignoring Honda decades earlier. Nvidia, like Honda, was today selling low-margin products to teenage boys, but if the analogy held, tomorrow they might overtake the business workstations of Sun Microsystems and SGI. Sometimes Jensen would even speak to his clustered executive staff about the possibility of disrupting Intel, then one of the most valuable firms on Earth. In the meantime, Nvidia would survive in Intel’s territory through a strategy of continuous tactical retreat. “To this day, we don’t compete with Intel,” Huang said in 2023, describing their Tom-and-Jerry relationship. “Whenever they come near us, I pick up my chips and run.” The gospel of Christensen counseled Nvidia to sell offbeat products that Intel would not conceive of making to customers it would never want to serve. “Jensen would tell us, even back then, that Nvidia could someday be bigger than Intel,” Kirk recalled. “It was just a question of strategy.

Kirk recalled showing off the Riva 128 to a competitor at a trade show. When the engineer saw what it could do, he gave up on the spot. “I hired him within a few days—and that killed that company, right?” Kirk said. “Because, you know, I removed its brain.” Kirk, mild and professorial, had a predator’s instincts. “We had all the geniuses from all the other start-ups, and as we were successful in overtaking more and more of these little companies, the remaining companies had a harder and harder time staying alive.

Huang educated himself by reading every business book he could find. “If you go into his office today, it’s totally abandoned; he never uses it,” one employee told me. “But it’s filled with stacks and stacks of business books.

Hans Mosesmann, a veteran industry analyst who would later help manage Nvidia’s IPO, recalled talking with one of Huang’s former managers at LSI who’d been tasked with giving Huang an employee evaluation. The evaluation form resembled a report card, but the manager left the grades blank. At the bottom, the manager wrote, “Jensen is an excellent employee. I look forward to working for him some day.

Decision-making, for Huang, was a clinical process with little room for useless emotions like hope. To him, business was just another engineering problem. Engineers looked to break down complex problems into simple governing principles, which could then be leveraged to powerful effect. To launch a start-up, then, Huang needed first to understand those principles. He had to research the market, the supply chain, the competition, the technology, and the product fit. He had to arrange exploratory phone calls with customers, game developers, and computer-graphics experts. He had to pore through years of dull state-of-the-industry reports, paging through bar charts and sales figures and customer surveys in search of a glimmer of an upward trend.

The ability of GPT-2 to answer new questions without explicit training was an example of the “emergent” properties in AI. These unexpected skills and behaviors were appearing as the models grew larger, surprising even the researchers. Once a model crossed the threshold of emergence, no one, not even its designers, could say what it was fully capable of doing.

Plenty of people worked long hours, though. Jens Horstmann attributed Huang’s success to his adaptability. “I’ve often asked myself, how is it that we started in the same cubicle, you know, with a similar IQ, both working equally hard,” Horstmann said. “How is it that this person not only built this amazing company, but also a network around him of people that—that would just die for him if needed?” Huang, Horstmann believed, had changed himself many times. He recalled Huang at LSI, pushing the simulation software to its outer limits. “Now, he’s still doing the same thing, but what he’s engineering is himself. He was not born as a great CEO; he was not destined to be one. He transformed himself into one, just by abstracting!

The reason that Nvidia succeeded was not that its circuits were better. The reason was that its software was better.

Between 2012 and 2022, Nvidia achieved a thousand-fold speed-up in single-chip AI inference performance, which was far in excess of anything that Moore’s Law had ever achieved. A mere 2.5× of that speed-up came from transistors; most of the remaining 400× came from Nvidia’s mathematical toolbox. “Honestly, AMD can make silicon just as good as we can,” Arjun Prabhu said. “They just can’t make the calculations go as fast.

In early 2024, an administrator at CalTech’s data center told me that the school’s wait time for delivery on an H100 chip was almost eighteen months. He had encouraged professors at the school to switch to other providers but found few willing to accept. “They’d rather wait for the hardware than switch away from CUDA,” he said. It was all this code that made Nvidia hard to compete against. Upstarts might design a new chip, but that wasn’t enough—Dwight Diercks, Nvidia’s head of software engineering, had ten thousand programmers working for him. “We’re really a software company; that’s the thing people don’t understand,” Diercks said.