Instacart Battles the ‘Brownie Recipe’ Problem with Modular AI

Article Summary

Instacart is grappling with the complexities of deploying LLMs in its real-time grocery delivery operations, moving beyond simple intent understanding to accommodate the nuances of the physical world. CTO Anirban Kundu describes this as the ‘brownie recipe problem’ – a LLM needs to move beyond a single request to understand local inventory, seasonal availability, and deliverability constraints to create a truly helpful experience. The company is employing a modular approach, dividing processing into foundational models for intent and categorization, alongside smaller language models (SLMs) for catalog context and semantic understanding. This architecture addresses the problem of ‘monolithic’ agent systems, acknowledging that a single AI could become unwieldy. Instacart’s strategy incorporates standards like OpenAI’s Model Context Protocol (MCP) and Google’s Universal Commerce Protocol (UCP) to manage interactions with diverse third-party systems. Despite these efforts, significant challenges remain, primarily around integration reliability and latency, with a considerable amount of time spent resolving error cases – indicating a continuous process of optimization and refinement. The company’s exploration of AI agents reflects a broader industry trend, prioritizing modularity and specialized tools over centralized, monolithic solutions.

Key Points

• LLMs struggle to handle real-world complexities like inventory and logistical constraints beyond simple requests.
• Instacart is adopting a modular AI architecture, splitting processing between foundational and smaller language models.
• Integration reliability and latency remain significant challenges, requiring substantial effort to resolve error cases.