Link

AI engineering, two years later

Context

Sam Bhagwat, part of the team that built Gatsby JS, is now building Mastra as a JavaScript/TypeScript AI framework.

His observation about the current state of AI engineering caught my attention because it perfectly captures where we are in 2025:

“Building AI applications today feels a bit like web development in 2012 - the primitives are powerful, but frameworks and best practices are still emerging.”

This comparison points to both the opportunity and challenge facing AI engineers today.

We have incredibly powerful tools, but turning them into production-ready systems requires navigating an evolving landscape of patterns and practices.

Key Implementation Patterns

The article reveals several emerging patterns in AI engineering:

1. From Single Prompts to Workflows

• Initial AI features were simple transformations
• Evolution toward graph-based data workflows
• Rise of “compound AI” for complex tasks

2. Knowledge Integration Pattern

• Combining general model knowledge with domain-specific data
• RAG (Retrieval-Augmented Generation) as the bridge
• Focus on chunking, embedding, and retrieval optimization

3. The Production Triangle

• Three key properties: accuracy, latency, cost
• Tracing for observability
• Evals as probabilistic testing frameworks

Understanding AI Agent Autonomy

The evolution of AI Agents follows a pattern similar to autonomous vehicles, progressing from basic automation to full autonomy.

At a basic level, we can think of this progression as:

• Low-level: Decision nodes in predefined workflows
• High-level: Self-directed task decomposition and execution

OpenAI has formalized this progression into five distinct stages:

1. Stage 1: “Chatbots, AI with conversational language”

   • Current state of most deployed systems
   • Basic interaction and response capabilities
   • Example: An AI system that mimics a human support agent in a website chat window

2. Stage 2: “Reasoners, human-level problem solving”

   • Systems that can break down complex problems
   • Apply logic and reasoning to reach conclusions
   • Example: An AI system diagnoses software bugs by analyzing error logs and suggesting specific fixes

3. Stage 3: “Agents, systems that can take actions”

   • Beyond reasoning to autonomous execution
   • Can make decisions and carry out tasks
   • Example: AI systems that manage cloud infrastructure, automatically scaling resources and handling failures

4. Stage 4: “Innovators, AI that can aid in invention”

   • Systems that can generate novel solutions
   • Contribute to creative problem-solving
   • Example: AI that can propose new software architectures or suggest innovative approaches to technical challenges

5. Stage 5: “Organizations, AI that can do the work of an organization”

   • Complete end-to-end process automation
   • Coordinate multiple agents and workflows
   • Think of it as an AI running an entire software development lifecycle: from requirements gathering to deployment and maintenance

Each stage builds upon the capabilities of the previous ones, creating a progression from basic interaction to full organizational autonomy.

Today, most production systems operate between stages 1 and 2, with some experimental systems reaching into stage 3.

Stages 4 and 5 remain largely aspirational but provide a roadmap for future development and serve as the ultimate goal of autonomous systems.

This leads us to consider the strategic implications for technical leaders…

Strategic Implications

Given the range of AI capabilities, from basic chatbots to aspirational organizational systems, technical leaders face both immediate challenges and future opportunities:

1. Time-to-Market Reality means managing stakeholder expectations around delivery points

• Short (1-2 days): Initial progress to get the “wow” factor
• Medium (1-2 weeks): Base demo to show organization the value it can drive
• Long (1-2 months): Production ready

2. Tooling Investment Strategy

• Invest in AI-enabled development environments
• Consider graph-based frameworks for complex workflows
• Build robust observability from day one
• Stay flexible as tools evolve and mature

3. Team Capability Building

• Focus on both prompt engineering and system design
• Develop new testing paradigms for non-deterministic systems
• Balance technical debt against rapid evolution

Implementation Framework

For teams building AI applications today, the key is matching implementation approach to your target autonomy level:

1. Start with Clear Foundations

• Choose AI-enabled development environments
• Select appropriate model providers
• Establish observability practices early

2. Build Progressive Complexity

• Begin with single-prompt transformations
• Graduate to multi-step workflows
• Introduce agents where appropriate

3. Focus on Production Readiness

• Implement comprehensive tracing
• Develop robust eval suites
• Monitor the accuracy-latency-cost triangle

Key Takeaways for AI Engineers

Overall

• Don’t commit too early to specific tools until you understand your use case
• Stay alert for new tools and techniques as the field evolves
• Work within LLM constraints while watching for ways to innovate
• Focus on solving real business problems rather than chasing the latest trends

Personal Notes

The pattern we’re seeing in AI engineering mirrors what I witnessed in the evolution of data science: an initial explosion of tools and approaches, followed by consolidation around solutions that actually solve real business problems.

I remember when it seemed like every week brought a new tool, technique, company, or buzzword in data science.

”What’s particularly interesting is that the ultimate winners weren’t the early movers, but rather the companies and tools that focused most intensely on solving real problems.

We’re witnessing the same pattern now with AI engineering in that a proliferation of tools and approaches arise every week and they will eventually consolidate around proven solutions.

Further, just like in the early data science days, the journey from “wow” to production is currently way longer than we’d like, but it’s a necessary evolution as we develop the patterns and practices that will define AI engineering for years to come.

For those building AI systems today, the key is maintaining balance: move fast enough to capture opportunities but build robust enough systems to survive contact with production.

Keep learning, keep delivering value, and don’t be afraid to adapt as better tools emerge.

Now is an exciting time to be in AI engineering.

Just as web development evolved from basic jQuery plugins to sophisticated frameworks like React and Angular, we’re watching AI engineering mature from simple prompts to complex autonomous systems.

While the landscape may seem chaotic, this chaos creates opportunities for innovation and improvement.

Mirroring the past of web development, the most successful tools and frameworks will be focused on solving pain points rather than chasing theoretical elegance.