AWS sits at the heart of the generative AI boom, powering everything from LLM training runs to global-scale inference. In this sweeping conversation, AWS CEO Matt Garman discusses the future of work, engineering, open vs. closed models, and why agentic workflows—not just raw models—will be where the next wave of value is created.

Garman is bullish on the economic upside of AI, skeptical of doomer narratives, and refreshingly candid about infrastructure bottlenecks, engineering culture, and how Amazon uses its own silicon to support customer choice.

01:00 – White Collar Bloodbath or Utopia?
Garman’s optimistic view on AI, jobs, and productivity.

04:15 – Hiring in the Age of AI
Why more productivity doesn’t mean fewer people.

08:52 – 80% of AWS Developers Use AI
How AI is changing developer workflows at Amazon.

12:55 – Should You Still Study Engineering?
Advice for the next generation in a fast-changing tech landscape.

15:46 – Infrastructure Bottlenecks
From silicon to power, what’s actually constraining AI growth?

18:12 – Where AI Usage Is Growing Most
Training matters, but inference drives demand.

20:05 – AWS Silicon Strategy
Graviton, Tranium, Inferentia, and why Annapurna matters.

27:57 – Serving the Model Ecosystem
Bedrock, specialization, and what AWS looks for in new models.

33:39 – Open vs. Closed Source
How AWS views the trade-offs and partnerships.

36:24 – Will AWS Build a Frontier Model?
On Nova, customer choice, and competition with partners.

41:33 – Benchmarks Are Breaking
Why standardized evaluations may not matter much longer.

47:13 – The Future of Agents
The biggest opportunity in AI—and how AWS is enabling it.

AI will remove toil, not jobs.

Productivity gains will fuel more opportunity—not layoffs.

AI coding tools are mainstream inside AWS.

But it’s about mindset, not memorizing syntax.

Today it’s silicon. Tomorrow? Could be power.

Training headlines. Inference dominates cost.

From Nitro to Graviton to Tranium, Annapurna made it all possible.

AWS isn’t betting on one model to rule them all.

The real value comes from customization.

Nova is the start—but partnerships will still thrive.

Models have learned how to ace them.

AWS is going all-in on agent infrastructure.

AI Will Enhance Work, Not Replace It
Garman sees AI as an augmentation tool—especially in creative, analytical, and engineering workflows.

Agent Infrastructure Is the Next Big Platform
From secure runtimes to memory and gateways, AWS is building the scaffolding for enterprise-scale agents.

Model Diversity > One-Model Dominance
Rather than an omni-model future, AWS is betting on specialization—offering customers choice through Bedrock.

Open vs. Closed Is a Spectrum
Customization, not philosophy, is what customers care about. Whether via open weights or closed APIs, it’s about fit.

Benchmarks Are Losing Signal
As models converge on test scores, real-world performance and UX integration matter more than academic leaderboards.

Silicon Strategy Is Core to AWS’s Edge
Owning the full stack—from chip to UI—gives AWS flexibility, pricing control, and customer alignment unmatched by competitors.

MB: I wanted to talk first about the future of work and white-collar work. There’s a spectrum—“white-collar bloodbath” to utopia. Where do you fall and why?

MG: I’m on the optimistic side. In tech there’s never been a more exciting time. Across industry, the advances in AI have enormous potential to increase efficiency, effectiveness, and enablement at work.

A lot of what AI promises is taking away the toil in day-to-day jobs. Today, the vast majority of time isn’t spent on what people get excited about—it’s not “put my numbers in this system,” “pull the report,” or “collate information.” That overhead takes a large percentage of time. AI can really help shrink that, letting people focus on the creative and analytical parts they love. That drives value for companies and people.

I’m very optimistic. This is not “no one has a job and robots run the world.” Companies and people become more efficient, and people spend more time on what they’re excited about.

MB: If you automate large swaths of tasks, you can do more as a company. Does that mean you won’t hire anymore? How do you think about hiring when every person is more productive?

MG: It depends—there’s no single answer for every company. Historically, efficiency gains come with a transition. The critical thing is for people to be flexible, willing to learn, and accept jobs will evolve. The job from two years ago won’t be identical two years from now.

There isn’t mass unemployment because we have computers, automation, or robotics. There are lots of jobs—often higher-paying. The economy is bigger; on average people are better off. Think of Excel: many used to spend time doing calculations. Excel didn’t eliminate those jobs; it changed them. AI is more disruptive than Excel, but the analogy holds: people move to higher-value work.

People are worried; I don’t minimize that. Embrace the technology. The more you do, the better off you’ll be. AI has potential to transform every industry, company, and job. It transforms, not replaces. If you don’t lean in, you might be out of a job; if you do, it makes you better/faster and lets you do more of what you like. That’s better for companies, people, and economic growth.

MB: Pessimists say AI’s speed is different from prior shifts. Will speed really affect the white-collar market?

MG: It’s a rapidly evolving space; people will have to move faster. For developers worried coding tools will make them unnecessary, I think we’ll need more developers, not fewer. The job changes: maybe you won’t author Java code; you’ll deconstruct problems, coordinate agents, and build systems.

The part you may not do in two or three years is authoring Java code—tools will be great at that. But pulling it together, reviewing, deciding it’s not quite right, coordinating agents—that becomes the developer’s job. That person drives more value. These tools unlock creativity: turning ideas into action takes time; if we unlock that, good developers become even more valuable.

MB: As a leader, if someone becomes 5–10x more productive, the last thing you want is fewer of them.

MG: Exactly. You invest more, because the ROI compounds. I don’t understand the math for wanting fewer of those people.

MB: How much of AWS’s code is written by AI? And define “written.”

MG: “Lines of code written by AI” is a silly metric—lines aren’t the point (fewer can be better). The last metric I saw: north of 80% of our developers use AI in their workflows—unit tests, docs, writing code, or agentic flows via things like QCLI or our Neuro IDE. That number goes up weekly.

MB: Are engineers proactively upskilling, or do you run programs?

MG: Both. Amazon’s large and not homogeneous, but most developers are curious. The number who haven’t used an AI coding tool rounds to zero. Using it to transform the job vs. using pieces—that’s where education matters. There’s a learning curve: how to change work, when it accelerates vs. slows you down.

First-gen tools can be linear: you “vibe code,” it gives you code, but it’s not what you want; hard to “go back.” With an agentic coding-first mentality, you start with a spec, then work with the tool to build parts of it. As you vibe code, it updates the spec—but the spec remains the source of truth you can modify.

This also guides junior developers toward best practices. Some leaders told me, “With AI we can replace all junior people.” That’s the dumbest thing I’ve ever heard—they’re least expensive, most leaned into AI tools, and you need a pipeline of talent. Keep hiring out of college; teach them to decompose problems and build software. Tools like Q can coach good practice and help juniors collaborate with seasoned engineers.

MB: Do you recommend engineering as a career to someone entering college?

MG: Yes—though kids should study what they’re passionate about. The emphasis should be: think for yourself, develop critical reasoning, creativity, and a learning mindset. With today’s pace, if you learn one thing and plan to ride it for 30 years, it won’t be valuable 30 years from now. Learn how to learn and how to think. Engineering is great for systems thinking and problem decomposition.

MB: Internally, how do you measure success using AI?

MG: We don’t have a magic new measure. Some of it is productivity. We encourage experimentation—tools, methods, setups. Previously, large systems needed many people focusing on pieces. Now, tools enable smaller, faster pods with broader scope. Startups move fast because of structure—big orgs can, too, if they organize as small pods. We’re leaning into that.

MB: Looking 2–5 years out: what are the bottlenecks—silicon, energy?

MG: Think of The Goal (book) on production lines: there’s always a bottleneck—solve it and the next appears. All of those (chips, power, networking) are bottlenecks at some point. Recent shortage: NVIDIA chips; as that eases, power could be next. It’s hard to get everything in sync at the growth rate we’re seeing; all require capital.

Our job is to think 1, 3, 5 years out: ensure enough power, capacity, and network for customers. Sometimes we’re wrong, sometimes we’re short—but we take that on so customers don’t have to.

MB: Where is growth—RL, inference, training?

MG: Inference drives most usage. Training and fine-tuning are important, but compute demand is dominated by the end-user interaction—questions, app usage, workflows. Infrastructure doesn’t care if it’s RL, FT, or inference—same silicon serves multiple needs (networking can differ for big pipelines). Tranium is great for training and inference (despite our naming); NVIDIA chips are similarly versatile.

MB: Custom silicon is a differentiator. What sets yours apart vs. Google TPUs?

MG: We start with customers: breadth of choice—capabilities, cost points, trade-offs. No single best solution for all workloads.

Timeline:

MB: Would you sell your chips to third parties (like rumors of Google selling TPUs)?

MG: Never say never. Today, selling only in AWS simplifies everything: one environment (AWS data center, server, network), fewer SKUs/firmware paths. Selling merchant silicon adds complexity—but it’s conceivable someday.

MB: Annapurna seems underappreciated. What did you see back then (pre-genAI)?

MG: They were mission-driven and matched Amazon’s culture. We wanted to offload virtualization to a card; nobody had that product. Annapurna was building a network card with ARM cores. We co-designed to also offload EBS virtualization. They were smart, scrappy, customer-focused, thought big. We acquired them—best acquisition we’ve made. Most of that team is still at Amazon 10 years later.

MB: When deciding which models to serve on Bedrock, how do you choose?

MG: We want choice: the best models customers might want. There’ll be a handful of frontier models (expensive), but lots of purpose-built ones. Recently we added Writer (agentic workflows) and 12 Labs (video understanding). We have Poolside (coding), Stability (image/video), Luma AI (video), and more. Ideally we’d support everyone—like AWS Marketplace. Some remain proprietary elsewhere, but our goal is breadth.

MB: Some predict one giant omni-model. I think we’re headed to specialization. Your view?

MG: From day one we believed customers will use many models. Today they do: a large general model for planning/reasoning; specialty models for specific workflows; fine-tuned models on proprietary data. Trade-offs on cost/capability lead to mixture-of-experts in practice.

Enterprises want models that deeply understand their data—better fine-tuning, domain knowledge, customer context. And as we move into an agent-focused world, the model is critical but not sufficient. You need scaffolding, workflows, memory, audit logs, and domain-specific pieces. Most ROI will come from agent workflows doing real work.

MB: You serve Anthropic (closed) and plenty of open-weight models. How do you think about open vs. closed—and partnerships with Anthropic, OpenAI, Meta?

MG: Most so-called “open” are open weights, not OSS. The key for customers is customization—bring your own data, tailor the model, run custom workflows. Whether via open weights (e.g., Llama, Mistral) or APIs for fine-tuning/distillation on “closed” models (e.g., Nova), the value is the same: fit to your use case. Over time, everyone will want customization; different vendors will enable it differently.

MB: Will AWS build a truly frontier model?

MG: We think choice matters. We’re investing in Nova to offer differentiated capabilities while maintaining deep partnerships (Anthropic, etc.). Compete where it helps customers; partner everywhere else—an AWS muscle we’ve built over 19 years.

MB: Pricing—will models race to cost of silicon + electricity?

MG: Unlikely. Models aren’t commodities today; why tomorrow? Cloud wasn’t commoditized after a year or two either. There are differences in availability, features, UX. Ask customers: Llama, Claude, GPT—are they commodities? No. Even open models differ (Mistral vs. Llama). Vendors must keep innovating; there’s real value to be captured.

MB: Open models seem ~3–6 months behind closed source. Will that persist?

MG: There’s no inherent advantage to open vs. closed; it’s a choice. China labs open-sourced their best while behind; that’s why some feel “open lags.” But models like DeepSeek and Qwen are impressive; customers love them. OpenAI released open models (smaller than GPT-5). Anthropic hasn’t open-sourced. Meta open-sources all Llama models. Whether they remain behind is about execution, not openness.

MB: Benchmarks are saturated—AM24, A2, MMLU. We’re down to single-digit gains. Do we need new benchmarks?

MG: Benchmarks are great for commodities (e.g., SSD throughput). The more complex the system, the worse benchmarks become. Early database benchmarks (TPC-C) faded; people test their own workloads. We’ll likely move the same way with models. It’s easy to train models to ace benchmarks; that doesn’t make them best overall.

MB: Your personal test?

MG: I prefer testing inside applications—how well it synthesizes research into coherent docs, generates ideas, interacts. Speed matters too. Integration/UX matters a lot—Perplexity is a good example with thoughtful UI and visible “thinking.” Latency is critical for real-time consumer use; for enterprise workflows (e.g., payroll agents), accuracy can trump speed and async is fine. We launched auditable reasoning (math-proof-like verification) applied to LLMs—powerful when you need correctness over latency.

MB: Where are you seeing repeatable, accurate agent implementations with positive ROI?

MG: Several:

With scaffolding easier, we see use cases across processing, individual productivity, marketing, sales, and industry workflows. Many are still human-in-the-loop, but the path to more autonomy is clear.

MB: Many worry about being automated away. Words of encouragement?

MG: Make yourself valuable. AI and agents amplify employees; they’re valuable through you. If you’re great at marketing, do marketing—not pulling campaign plumbing. If you’re great at building apps, be great at building, not at memorizing a language. Learn the tools, focus on customer problems, keep learning. I have very little worry that jobs just disappear and robots run everything.

MB: Thank you, Matt.
MG: Thank you—this was really fun

Follow us on X and subscribe to our YouTube channel for more interviews with the people building the future of AI.