Résumé
All writing

8 min read

Iron Man’s Jarvis Is Almost Ready — And It Explains the Evolution of AI

AI Agents · LLMs · Machine Learning · AI

When I first started my AI journey, shaving half a point off the error rate on MNIST was a legitimate research event. Many people built careers on incremental gains against benchmarks most outsiders had never heard of. Today, the dominant way to “do AI” is to open an API key, wire an existing model into a business workflow, and ship. Same field. Almost an unrecognizable profession.

What practicing AI used to require

The way (a+b)² = a² + 2ab + b² is drilled into every algebra student, y = mx + b was drilled into every AI practitioner. Linear regression is the first thing anyone learns in this field. For a long time, a huge amount of real work never went much further than variations on it — fit a line, fit a curve, fit a decision boundary. Even now, financial services, healthcare, supply chain, retail, and energy still rely heavily on linear regression.

Neural networks changed what was possible, but they came with a price: backpropagation. I remember doing math for days before writing a single line of training code — deriving the chain rule by hand across every layer, working through why a particular gradient update was stable before trusting it enough to actually run. Only after that did the coding start, and even then you were building the model itself: reading research papers, choosing an architecture, engineering features by hand, tuning a training loop that might run for days to weeks before you knew if it worked.

Understanding backpropagation properly took me the better part of a week — chasing down papers, working through derivations, piecing together a real understanding from sources that were never written to teach a beginner. Today you can open a chatbot, ask it to walk you through backpropagation step by step, and have a working derivation in front of you in minutes. It’s not just that the work got easier. The barrier to learning the work nearly disappeared too.

Progress was measured in benchmarks. MNIST, then ImageNet, then whatever came next. A fractional accuracy improvement was worth publishing. That was the job: build a model, one problem at a time, from something close to first principles.

Math didn’t disappear. It got compiled away.

Here’s the part I think is actually the throughline of the whole story, not just a nostalgic detail: every layer of progress in this field has worked the same way. Something that used to require deep, hard-won expertise gets absorbed into a tool, and the next generation of practitioners doesn’t have to relearn it from scratch.

Hand-derived gradients became autodiff — PyTorch and TensorFlow started doing the calculus for you. Hand-built architectures became pretrained models — transfer learning did the design for you. Pretrained models became foundation model APIs — a single API call did the training for you. And now, foundation models are becoming agentic systems — a framework does the orchestration plumbing for you.

Nobody derives gradients by hand anymore, yet understanding them remains invaluable when training becomes unstable or a model behaves unexpectedly. Infrastructure replaced the repetition, not the understanding.

Fig 1 — Timeline for evolution of abstraction

The abstraction didn’t just change how we build AI — it changed how we learn it. That’s a strange kind of recursion: the tool built from decades of hard-won expertise is now the fastest way to acquire that expertise in the first place. Each step didn’t make the field shallower. It moved the floor up and freed practitioners to work one layer higher than before. That’s the strange beauty of building something powerful enough: it eventually makes the struggle to build it unnecessary for the next person.

Two breakthroughs that changed everything

Two pieces of research stand out for changing the course of AI. The first is XGBoost. When it showed up in 2014, gradient-boosted trees weren’t new, but XGBoost was fast and effective. It became dominant for years — it was hard to find a winning Kaggle solution on structured, tabular data that didn’t use it. For a huge share of real business problems, XGBoost was about as state-of-the-art as it got. Like autodiff, it turned years of optimization research into a single library call — proof that strong engineering and an optimized implementation could outperform a more complex idea, not just a bigger one. It was the workhorse of an entire era of practical machine learning. Even today, many production systems use it.

The second is the one that actually broke the field open: “Attention Is All You Need,” published in 2017. Before it, sequence models were built on recurrence — architectures like RNNs and LSTMs that processed one step at a time. These structures made training slow and hard to parallelize. The Transformer architecture that paper introduced replaced recurrence with self-attention — letting a model weigh every part of a sequence against every other part at once, in parallel.

That single architectural change is the reason large language models exist at the scale they do. Parallelizable training meant you could throw enormously more compute and data at a model than a recurrent architecture could ever practically absorb. XGBoost proved you could win without deep learning. Attention Is All You Need proved you could go bigger with it than anyone had realistically planned for.

All of these findings are the reasons we’re where we are now: foundation models became good enough that the interesting work moved from building them to orchestrating them.

The unit of work changed

The shift can be summarized in one sentence: it moved from training to orchestrating. Almost nobody is training a new classifier from scratch to route support tickets anymore. They’re taking an existing LLM and shaping its behavior with prompts, retrieval, and tool access — deciding what it can see, what it can do, and when it should ask for more information instead of guessing.

That’s the entire premise behind agentic RAG, which I’ve been writing about in this series: you’re not training anything. You’re building a loop around a model that already exists — deciding when it should retrieve more context, when it should validate what it found, when it should try again. The skill moved from model-building to system-building.

The warning underneath the opportunity

Here’s where I think people get this wrong: they see “prompt engineering” and “agentic building” exploding in demand and assume that’s the skill to chase. I’d be careful. It reminds me of the years when everyone wanted to study IT because the demand was high — until the demand caught up with the supply, and just knowing how to use a computer stopped being a differentiator at all.

If history repeats itself, building agentic applications is likely heading toward the same fate. It’s valuable right now — for many people, “doing AI” simply means developing agentic applications. But I don’t think that advantage will last. Frameworks are already absorbing more of the orchestration work, just as autodiff absorbed the calculus. The skill that feels essential today is only one abstraction layer away from becoming a platform feature.

The fundamentals didn’t vanish. They’re what separates someone who can debug a broken system from someone who can only assemble one when the framework’s defaults stop working. I believe agentic-building skill is about to split the same way. The engineers who only know how to wire a framework together will be exposed the moment the framework gets good enough to do it for them. The engineers who understand what’s happening underneath will be the ones who can move to whatever the next layer turns out to be. This is where I believe companies hiring for such engineers should focus.

The brain

Whenever I think about where all this is heading, I end up thinking about Iron Man’s Jarvis — not as a metaphor, but as an actual technical comparison. I used to always wonder how magical Jarvis felt. Maybe what made Jarvis feel like science fiction when I was growing up wasn’t that he could talk — it was how he could reason, plan across steps, use tools, understand context, and correct himself when he got something wrong.

In many ways, we already have that. LLMs can do most of what made Jarvis feel impossible — the core reasoning capability is basically here. I saw a post saying Iron Man is just someone with unlimited Claude tokens — with what we’re getting to experience, I wouldn’t deny that. What’s missing now is a body: natural voice instead of typed text, persistent memory instead of a fresh context window every time.

That’s why “agentic” is the right word for this exact moment in the field. It’s not a new kind of intelligence but an assembly. Which is also why the voice AI numbers happening right now aren’t really a side story — they’re the last major component of Jarvis getting built in real time.

Big tech is racing on this too — multilingual voice, personalized assistants tuned to each user — and the money backs up the urgency. Voice AI funding surged roughly eightfold in a single year, reaching around $2.1 billion in 2025. Nearly 80% of the world’s top 50 banks are already running production voice agents, up from about a third of them just two years ago. And almost a quarter of Y Combinator’s latest cohort is building voice-first rather than text-first products.

Text in, text out was never the endpoint. It was simply the interface that happened to be easiest to ship first. The gap between “chatbot” and “Jarvis” was never really about reasoning capability anymore — it’s the interaction layer, and that’s exactly where the industry’s attention is moving now.

Where that leaves you

If you’re building a career in this field right now, I’d take two things from this. First: the market is real, and applied, system-building AI skills are genuinely in demand — that part isn’t hype. Second: don’t mistake “in demand today” for “durable.” The specific skill of wiring an existing model into a workflow is going to keep getting easier and more automated, the same way training a model got easier and more automated before it.

The practitioners who’ll still matter once that happens aren’t the ones who only learned to assemble agentic applications. They’re the ones who paired that skill with an actual understanding of the fundamentals underneath it — the same fundamentals I spent days grinding through math for, years ago, before I ever got to write a line of training code. That foundation is what lets you move to whatever the next layer turns out to be, instead of getting stranded when this one gets automated too.

Looking back, shaving half a point off MNIST wasn’t really the point. If history keeps repeating, the systems we’re painstakingly wiring together today will eventually become someone else’s API call.

Jarvis was a fictional character when I was growing up. Watching this field move as fast as it has, that fiction feels closer than it’s ever been — not because the brain got smarter overnight, but because we’re finally building everything around the brain.

Sources