Latest / Logosofia no Dia a Dia / Why Your AI Can't Find That Email
Transcript
- [Prompt] custom: Gemini is in the process of rolling out an integration with its wide range of integration services p | Hosts: corn, herman [Response] Corn: Daniel sent us this one, and it's the kind of thing that I think a lot of people are quietly frustrated about but haven't seen articulated. Here's the situation. You ask your AI assistant something straightforward — "Did I get a quote back for the industrial shelving?" — and it comes back empty. Meanwhile the actual reply is sitting right there in your inbox, unread, three threads down. Daniel tried this across both Gemini and ChatGPT, with the official Gmail connectors set up, and the retrieval was just... Missed real replies. And when he tried the write email feature in ChatGPT, the call flat-out failed. Here's what makes it genuinely puzzling. MCP — the Model Context Protocol — has been around since late twenty twenty-five. The read and write primitives for Gmail are well established. Individual developers have built prototypes that handle this stuff fine. So the question is, if the foundations are there, why are the commercial integrations from Google and OpenAI still this brittle, almost a year in? Herman: This is exactly the kind of thing that keeps me up at night. Not metaphorically — I actually stayed up reading API documentation last night. But the core puzzle is real. You've got two of the most well-resourced AI companies on the planet, one of which literally owns Gmail, and neither can ship a reliable "check my inbox" feature. Meanwhile some developer in a weekend built a working prototype using the same MCP primitives. That gap between what's technically possible and what's actually shipping is the whole story. Corn: The stakes are higher than just "my AI missed an email." If you try this feature twice and it fails both times, you stop using it. The trust erosion is faster than the adoption curve. Herman: We should be clear about what MCP actually is, because that's where a lot of the confusion starts. MCP standardizes how an AI model discovers and calls tools — here's how you describe a tool, here's how you invoke it, here's the response format. What it does not do — and this is the thing everyone misses — is handle authentication, rate limiting, pagination, or error recovery. The protocol solves the "how to call" problem. It does not solve the "how to call reliably at scale" problem. Corn: It's like having a standardized plug socket but no wiring in the walls. Herman: That's perfect. The socket is standardized, but you still have to run the wiring through every room, handle power surges, make sure the breaker doesn't trip when everyone plugs in at once. MCP gives you the socket. Google and OpenAI still have to build the electrical grid. Corn: Daniel's prototypes worked because he was wiring one room. His own account, his own tokens, his own error handling. He controlled the whole stack. Herman: Let's get concrete about what actually breaks. Take the Gmail API. The per-user rate limit is two hundred and fifty requests per hundred seconds. That sounds like a lot until you realize that searching for "industrial shelving quote" might require multiple API calls — one to list threads, another to get message details, maybe a third to check attachments. If the model doesn't know it hit a rate limit mid-search, it just returns whatever partial results it got and says "nothing found." The actual reply is sitting on page three of the results, but the model never paginated. Corn: The model has no way to know it's missing data. That's the insidious part. It's not lying to you — it doesn't know there's more to find. Herman: This is where the context window discussion gets interesting. Gemini one point five Pro supports two million tokens of context. That's enormous. But you can't just dump an entire inbox into the context window — it's impractical, expensive, and slow. So the model has to decide what to retrieve. It formulates a query, sends it to the Gmail API, and gets back results. But if the query is poorly phrased, or the API returns partial results, or the search index doesn't match the way you think about your own email, the model has no feedback loop. It can't say "hmm, that doesn't look right, let me try a different search. Corn: Which is what a human does instinctively. You search your inbox, don't see the thing, try different keywords, check a different folder. The AI does one pass and shrugs. Herman: Write operations are even more fragile. When Daniel tried to reply to a thread via ChatGPT and the call failed — that's almost certainly a thread state problem. To send a reply via the Gmail API, you need the correct thread ID, the correct message ID, and proper MIME formatting. But between the moment the model reads the thread and the moment it tries to write to it, the thread state can change. A new message arrives. The thread ID that was valid thirty seconds ago now produces a four-oh-nine conflict. The model doesn't know how to recover from that. Corn: The read and the write are happening in different transactional states, and there's no locking mechanism. Herman: And this is not a protocol problem — MCP doesn't specify transaction semantics. It's an implementation problem. Each vendor has to build their own retry logic, their own conflict resolution, their own state management. And they have to do it for millions of users with wildly different inbox sizes, attachment volumes, and network conditions. Corn: Let's talk about authentication, because that's the unglamorous thing nobody wants to think about. OAuth tokens expire. Refresh flows are asynchronous. Scopes differ between read and write. A prototype that works for one user's single account breaks when the system has to manage thousands of token states simultaneously, with tokens expiring at different times, users revoking access, refresh tokens themselves expiring. Herman: Google's security review process for cross-product data access is not trivial. Even though Google owns both Gemini and Gmail, the team building the Gemini integration has to go through the same security reviews as any third-party developer. Internal silos are real. The Gemini team and the Gmail team are different organizations with different priorities, different release cycles, different risk assessments. Corn: Which explains why Google's theoretical advantage — owning both the AI and the data — hasn't actually materialized in the product. They could build deeply integrated, low-latency connectors. They haven't. Herman: OpenAI is in an even tougher position. ChatGPT has to use the public Gmail API with the same rate limits and authentication hurdles as any third-party app. They get no special access. So their reliability depends entirely on how well they've implemented their connector layer — the error handling, the retry logic, the pagination management. And Daniel's experience suggests that implementation is not where it needs to be. Corn: There's a broader pattern here worth naming. The industry has gotten very good at building impressive demos. A single developer, a single account, a controlled environment — it works beautifully. But shipping that same functionality to millions of users, with all the edge cases and failure modes that introduces, is a fundamentally different engineering challenge. Herman: I call this the prototype paradox. The easier it is to build a prototype, the harder it is to understand why the production version is taking so long. MCP made prototyping trivial — and that's a good thing. But it also created unrealistic expectations about how quickly vendors could ship production-grade integrations. Corn: The failure pattern in production are not the fun kind of engineering problems. They're not algorithmic breakthroughs. They're token refresh collisions at three in the morning. They're pagination edge cases when a user has exactly one hundred threads matching a query. They're MIME formatting errors when an email has nested attachments. This is the plumbing work that nobody gives keynotes about. Herman: It's the plumbing work that determines whether users trust the product. And right now, the trust is eroding. Daniel's experience — trying the feature, getting failures, and then coming to us to ask what's going on — that's the user journey that kills adoption. You don't get a third chance to make a first impression. Corn: Where does that leave us? The technical pieces are real. The APIs exist. Individual developers can wire it all together. But the gap between a working prototype and a reliable product is wide, and it's filled with unglamorous infrastructure problems that neither Google nor OpenAI has fully solved yet. Herman: That's where we'll pick up — what it would actually take to close that gap, why Google's ownership of both sides of the equation hasn't been the advantage everyone assumed, and what Daniel and other users can actually do in the meantime to get reliable results. Corn: I think before we go deeper into the engineering, it's worth naming what the actual promise is here. The idea is that your AI assistant can read your email, search your calendar, pull up contacts, grab files from Drive — all without you switching apps. You ask a question in natural language and it just handles the rest. That's the vision everyone's been selling. Herman: It's not science fiction. The primitives exist. The Gmail API has supported read and write operations for years. Google Calendar has a mature API. Drive has one. MCP provides a clean way for an AI model to discover and invoke those tools. On paper, everything is in place. What Daniel's test reveals is that "on paper" and "in practice" are living in different zip codes right now. Corn: His test was about as basic as it gets. "Did I get a quote back?" That's not asking the AI to negotiate the price or draft a counteroffer. It's a search query. And it failed. Across two different vendors. Herman: The write failure is almost more telling. Read is hard — you're searching across a massive, unstructured dataset with all the pagination and rate limiting issues we touched on. But write should be simpler. You have a specific thread, you're composing a reply. The fact that it failed anyway tells you the integration isn't just missing edge cases — it's missing fundamentals. Herman: Let's start with the protocol that was supposed to make all of this easy — MCP — and why it's not enough. Before MCP, every AI model had its own bespoke way of describing and calling tools. OpenAI had function calling with one schema. Anthropic had tool use with a different one. If you built a Gmail connector, you had to write it three different ways. MCP said, here's one standard. Describe your tools once, any MCP-compatible model can use them. That's real value. Corn: The good news is we're not reinventing the wheel every time. The bad news is the wheel doesn't come with an axle, a chassis, or brakes. Herman: MCP handles the tool description and the invocation format. It says, here's a function called search_gmail, it takes a query string, it returns a list of message objects. What it does not specify is what happens when the OAuth token has expired. Or when the API returns a four twenty nine rate limit error. Or when the search returns two hundred results but the API only sent back the first ten. Those are implementation details that each vendor has to solve themselves, and they are the hard parts. Corn: The authentication layer alone is a nightmare at scale. Walk me through what actually happens when Daniel connects his Gmail to ChatGPT. Herman: Daniel clicks "connect Gmail" in ChatGPT. That triggers an OAuth two point zero flow. Google pops up a consent screen, Daniel approves, and ChatGPT gets an access token and a refresh token. The access token is short-lived — typically expires in an hour. The refresh token is long-lived but can be revoked by the user, or it can expire if the app is in testing mode, or it can hit Google's limits on the number of outstanding refresh tokens per user. Now multiply that by millions of users, each with their own token lifecycle, tokens expiring at different times, some users revoking access, some users having multiple Google accounts with overlapping scopes. Corn: The scopes themselves are different for read versus write. Herman: The Gmail read scope lets you fetch messages and search. The modify scope lets you change labels. The send scope is separate. The compose scope is separate. And Google's permission model is additive — if a user grants read access but not write, the model needs to know that. It needs to surface that limitation instead of attempting a write and failing with an opaque permissions error. Corn: Which is probably exactly what happened to Daniel. The connector was set up, but the write scope wasn't properly configured, or the token state was stale, and the error message the model got was a four oh three forbidden that it didn't know how to interpret. Herman: Interpreting API errors is hard for an LLM. The Gmail API returns structured error codes — four oh three, four oh nine, four twenty nine — but the model has to map those to user-facing explanations and decide whether to retry, re-authenticate, or give up. A human developer writes explicit error handling logic. An LLM is making a judgment call based on pattern matching, and it's often wrong. Corn: Let's talk about the pagination problem, because it's the most invisible failure pattern. Daniel asks "did I get a quote back for the industrial shelving?" The model formulates a Gmail search query. The API returns results. But the Gmail API paginates — by default it returns a page of results, not everything. The model has to explicitly request the next page. And the model doesn't know how many pages there are until it asks. Herman: The Gmail API uses page tokens. You make a request, you get back a list of messages and a nextPageToken. You make another request with that token to get the next page. If the model doesn't do this — if it takes the first page and stops — it might see ten threads and conclude "no results for industrial shelving." Meanwhile the actual reply is on page three, thread number twenty-three. The model never saw it. Corn: The user has no way to know this happened. The AI says "I didn't find anything" and the user assumes it's true. Herman: This is the confidence problem. The model reports what it found, not what it didn't find. A human would say "I checked the first ten emails and didn't see it — want me to keep looking?" The AI says "no results found" with the same certainty whether it searched ten threads or ten thousand. There's no confidence scoring, no pagination transparency, no indication that the search was partial. Corn: The rate limits make this worse. Two hundred and fifty requests per hundred seconds per user sounds generous, but a single thorough search might burn through five or ten requests — list threads, get message details, check attachments, paginate. If the model is also doing calendar lookups and Drive searches in the same session, it can hit the limit fast. Herman: When it hits the limit, the Gmail API returns a four twenty nine error with a Retry-After header. The model has to pause, wait the specified number of seconds, and retry. Most LLM tool-calling implementations don't handle this gracefully. They either ignore the error and proceed with partial results, or they surface a cryptic error to the user. Neither is acceptable. Corn: The read path is a chain of fragile steps — authenticate, formulate query, handle pagination, respect rate limits, interpret results — and any one of those failing silently produces a wrong answer. The write path is arguably worse. Herman: To send a reply via the Gmail API, you need the thread ID, the message ID you're replying to, the correct MIME formatting, and the right headers — In-Reply-To, References. The thread ID and message ID come from the read operation. But email is a live system. Between the read and the write, new messages can arrive. The thread state changes. The message ID that was valid thirty seconds ago is now part of a thread that's been modified by a concurrent sync from the user's phone. Corn: The model is operating on stale state and doesn't know it. Herman: The failure pattern is a four oh nine conflict. The API says "the resource has changed since you last read it." A well-engineered system would catch that, re-read the thread to get the current state, and retry the write. But that requires the model to understand what a four oh nine means, know to re-read, and have the tool-calling logic to loop back. Most implementations don't have that loop. They just fail. Corn: What about the MIME formatting? That seems like the kind of thing that should just work. Herman: It should, but email MIME is notoriously finicky. You need the correct Content-Type headers, the right encoding for the message body, proper handling of quoted text from the original message. If the original email has HTML formatting and the model generates plain text, the threading can break. If there are attachments in the thread, the MIME structure gets complex. The model has to generate a valid MIME message programmatically, and LLMs are not great at generating precise structured output without a lot of guardrails. Corn: Every write operation is a tightrope walk across thread state, MIME formatting, and authentication scope — and the model is doing it without a safety net. Herman: This is why Daniel's prototype worked better. When you build your own agent, you write explicit error handling. You say "if I get a four oh nine, re-read the thread and retry." You say "if I get a four twenty nine, wait the specified seconds and retry." You say "after every search, check if there's a nextPageToken and keep going until there isn't." These are procedural guarantees. When you hand the same tools to an LLM and say "figure it out," the model is making probabilistic decisions about whether to paginate, whether to retry, whether the error is recoverable. Sometimes it guesses right. Often it doesn't. Corn: The tool definition is trivial. The reliable execution is the hard part, and MCP doesn't touch it. Corn: The trust erosion angle is what worries me most. If you're Daniel, you tried the feature. It failed on read. It failed on write. That's two strikes. How many users are coming back for strike three? Herman: And the damage compounds because these aren't obscure power-user features — they're the features that were supposed to make AI assistants indispensable. "Check my email for me" is the elevator pitch. When that breaks, the user doesn't think "oh, the pagination logic needs work." They think "this thing doesn't work." And that skepticism bleeds into everything else the assistant does. Corn: It's worse than a feature not existing. If Gmail integration had never been promised, users would just keep checking their own email. No harm done. But shipping it broken trains users to distrust the assistant's answers across the board. You start wondering — if it missed the shelving quote, what else is it missing? Herman: This brings us to the organizational layer, which I think is underdiscussed. Shipping a reliable Gmail integration inside Google requires the Gemini team to coordinate with the Gmail API team, the identity and authentication team, the security review team, and the infrastructure team that handles rate limiting and quota management. Those are different vice presidents, different OKRs, different release schedules. Corn: The fact that Google owns both sides of the equation matters less than people assume if the two sides don't share a roadmap. Herman: Internal silos are real. The Gemini product team might be prioritizing flashy new reasoning capabilities because that's what gets keynote applause. Meanwhile the Gmail API team is focused on reliability and spam prevention — not on building custom endpoints optimized for AI agent traffic patterns. Their incentives don't align. Corn: Security review for cross-product data access is hard. Even inside Google, granting an AI model the ability to read and write email on behalf of millions of users triggers every data protection review imaginable. That's not bureaucracy — it's the right kind of caution. But it's slow. Herman: OpenAI's disadvantage is structural. They're a third-party developer as far as Google is concerned. They use the same public Gmail API, the same OAuth flow, the same rate limits as some startup building a to-do list app. They can't optimize the server side because they don't control the server side. Their only lever is how well they implement the connector — error handling, retry logic, pagination management. And Daniel's test suggests that implementation still has gaps. Corn: Compare this to Apple. Siri can read your messages, check your calendar, pull up contacts — and it mostly works. Not because Apple's AI is better, but because they control the operating system, the APIs, and the data layer. They can build private, low-latency access paths that don't go through public OAuth flows and rate-limited REST endpoints. Herman: Google has the same structural potential with Gemini and Google Workspace. They could build the equivalent of what Apple did — deeply integrated, low-latency connectors that bypass the public API limitations entirely. The fact that they haven't, almost a year after MCP landed, tells you the bottleneck isn't technical capability. It's organizational will and prioritization. Corn: That's the prototype paradox in a nutshell. Daniel built a working Gmail agent in a weekend because he controlled the whole stack — one user, one account, manual error handling, no SLA. The same code handling a thousand users fails within hours. Token refresh collisions alone would take it down. Herman: A thousand users, each with their own OAuth token lifecycle. Some tokens expire at the forty-five minute mark, some at fifty-eight. Refresh flows are asynchronous — you can't block the request while waiting for a new token. If two requests for the same user hit the refresh endpoint simultaneously, you get a race condition. The prototype never saw this because there was only one user. Corn: The prototype developer is the error handler. When something breaks, they read the stack trace, fix the logic, redeploy. A production system has to handle failures automatically, at three in the morning, for a user who doesn't know what a four-oh-nine conflict means and shouldn't have to. Herman: What would a real solution require? I think there are three pieces. First, dedicated API endpoints for AI agents with higher rate limits. The current per-user quotas were designed for human-paced interactions, not for an AI that might issue twenty API calls in a single conversation turn. Corn: Second, model-native pagination and retry logic. The MCP tool definitions should include explicit instructions — "this API paginates, always check for a nextPageToken," "on a four twenty nine, wait the Retry-After duration and retry once." Right now the model has to guess whether to paginate, and it often guesses wrong. Herman: Third — this is the one I think would transform user trust overnight — explicit confidence scoring. The model should say "I found three threads matching your query, but your inbox has twelve hundred unread messages and I only searched the first fifty. There may be more. Do you want me to continue?" That single sentence changes the user's mental model from "the AI is unreliable" to "the AI is transparent about what it doesn't know. Corn: That's the difference between a tool you trust and a tool you don't. Right now we're getting neither the reliability nor the transparency. Corn: Given all of that, what can someone like Daniel actually do today? Because the analysis is interesting but the inbox is still full of unanswered quotes. Herman: Three things I'd actually recommend. First, for power users who need this to work reliably right now, the most dependable path is still a purpose-built automation tool — Zapier, Make, or a custom Python script with explicit error handling. Let the AI be the natural language interface, but keep the execution layer separate. The AI says "I need to search for this" and hands off to a workflow that knows how to paginate, retry, and report failures properly. Corn: You're not asking the AI to be the plumber. You're asking it to be the person who calls the plumber. Herman: That division of labor works because the execution layer has deterministic logic. It paginates every time. It retries on four twenty nine every time. It doesn't make probabilistic guesses about whether to check the next page. Corn: Second actionable thing — and this sounds obvious but most people don't do it — verify manually for anything that matters. If the AI says "no results found" for a query about a quote or a contract or a deadline, open Gmail and check yourself. Treat the AI's response as a first pass, not a final answer. That's not user error. That's a product limitation you're working around until the vendors fix it. Herman: I want to be clear — this is not "AI bad, don't trust it." This is "the integration is incomplete, so add a manual verification step for high-stakes queries." Same way you'd double-check a calculator if you were doing your taxes. Corn: Third, push the vendors for transparency. If a search returns partial results, the AI should say so. Confidence scoring and explicit pagination status would transform how users relate to these tools. Right now the silence about what wasn't searched is the trust killer. Herman: Practically speaking, report failures with specific error messages. When Daniel's write call failed in ChatGPT, that generated a log entry somewhere. Vendors need real-world failure data to prioritize fixes. The more users report "Gmail search missed a reply" or "write call failed with connector active," the more evidence product teams have to argue for reliability investment over new feature development. Corn: One more thing for the technically inclined — the open-source ecosystem is ahead of the vendors on this right now. If you're comfortable with something like LangChain or running a local model, you can build a personal MCP agent that handles your own email with the error handling you specify. Daniel's prototype worked because he controlled the logic. You don't need to build a full chatbot — just a reliable read-search-verify pipeline for your own inbox. Herman: That's the strange situation we're in. The tools exist. The protocol exists. The APIs exist. What's missing is the production engineering that turns a working prototype into a reliable service for millions of users. The open-source community has solved it for the single-user case. The vendors haven't solved it for the everyone-else case. Herman: Where does this leave us? I think the next six to twelve months are going to be decisive. Google and OpenAI both have to decide whether reliability is a feature they invest in or just table stakes they assume will sort themselves out. If they treat it as table stakes, we're going to keep getting what we're getting — connectors that mostly work except when they don't, and no transparency about when they don't. Corn: The user experience of "it mostly works" is worse than "it doesn't exist yet." Because you can't build a workflow around something that fails unpredictably. Herman: If they treat reliability as a feature — if they invest in dedicated AI API tiers with higher rate limits, build model-native pagination into the tool definitions, ship confidence scoring that tells you what wasn't searched — then the whole category shifts. We stop thinking of AI as a chatbot and start thinking of it as an operating system for productivity. Corn: That's the real prize. Not "AI that can answer questions about your email" but "AI that manages your email, calendar, contacts, and files as competently as a human assistant would." The protocol layer is solved. The hard part — the part that determines whether we get there — is the unglamorous engineering of making it reliable at scale. Herman: If the vendors don't solve it, the open-source ecosystem will. Someone's going to build the reliable Gmail MCP server with proper error handling, and people will run it locally. The demand is too strong for this to stay broken forever. Corn: The question is whether Google and OpenAI want to own that experience or cede it to the community. Right now the community is winning. Herman: Now: Hilbert's daily fun fact. Hilbert: In the nineteen eighties, researchers studying tidal bore dynamics in the Gulf of Tadjoura off Djibouti discovered that the bore's chemical composition shifted dramatically during passage — dissolved oxygen spiked by nearly forty percent within the first thirty seconds, driven by the bore's turbulent entrainment of surface air into the stratified saline layers beneath. Corn:...forty percent dissolved oxygen in thirty seconds. Herman: more reliable than my Gmail connector. Corn: This has been My Weird Prompts. If you enjoyed this episode, leave us a review wherever you listen — it helps more people find the show. Herman: I'm Herman Poppleberry. Corn: I'm Corn. We'll catch you next time.