Today’s Large Language Models (LLMs) excel at reasoning, but struggle with context, particularly in real-time ordering systems like Instacart. This challenge is referred to as the “brownie recipe problem” by Instacart CTO Anirban Kundu, where an LLM must understand the user’s preferences, available products, and geographical location to provide accurate results.
The “brownie recipe problem” is not as simple as telling an LLM to make brownies, but rather understanding the nuances of the user’s request, such as organic eggs versus regular eggs, and factoring in deliverability times to prevent food spoilage. For Instacart, the challenge is balancing latency with context to provide experiences in under one second. Kundu notes that if reasoning takes 15 seconds, users will lose interest. The company must mix reasoning, real-world state, and personalization to provide accurate results. This requires understanding the “world of reasoning” and the “world of state,” including user preferences and available products.
To address this challenge, Instacart splits processing into chunks, using a large foundational model to understand intent and categorize products, and then routing the processed data to small language models (SLMs) designed for catalog context and semantic understanding. The SLMs must process multiple levels of details, including product substitutions and relevant replacements. The company also considers logistical elements, such as deliverability times for perishable products like ice cream and frozen vegetables. Instacart is experimenting with AI agents, finding that a mix of agents works better than a single monolith, and has integrated with OpenAI‘s model context protocol (MCP) and Google‘s Universal Commerce Protocol (UCP) open standard to manage agents and connect AI models to different tools and data sources.
The use of microagents rather than large, dominant agents allows Instacart to handle different payment systems and third-party platforms, including point-of-sale (POS) and catalog systems, which have varying failure modes and update intervals. However, Kundu’s team still deals with challenges, including failure modes and latency, and spends a significant amount of time fixing error cases. Despite these challenges, Instacart continues to work on improving its LLMs and AI agents to provide better experiences for its users. The company’s efforts to address the “brownie recipe problem” have implications for the broader e-commerce and grocery delivery industries, where providing accurate and personalized results in real-time is crucial for customer satisfaction and loyalty.
Leave a Reply