Gemini 1.5 Pro launched with a 1M token context window that generated significant coverage. Six months into real production deployments, the picture is more nuanced: long context works well for a specific class of problems, performs worse than retrieval for others, and costs significantly more than most teams modeled.
Where it works: whole-codebase reasoning tasks, long legal document analysis, and multi-document synthesis where the semantic relationships span the entire document set. These workloads benefit from the model having the full context in working memory rather than relying on retrieval to surface relevant chunks. Teams doing code archaeology on large repos report genuine quality improvements over retrieval-augmented approaches.
Where it underperforms: question-answering over structured knowledge bases, customer support on bounded domain documents, and any workload where the relevant context is a small fraction of what you’d feed in. For these workloads, a well-tuned retrieval pipeline with a 32K context model consistently beats 1M context at a fraction of the cost.
The cost math at 1M tokens: Google prices input tokens at approximately $7/million for Gemini 1.5 Pro (long context). A single 1M token call costs around $7 in input. For applications doing dozens to hundreds of calls per user session, that arithmetic requires careful unit economics work before you commit to the architecture.
The practical recommendation: start with retrieval. Escalate to long-context for the specific workloads where retrieval demonstrably fails on quality — not on convenience. The architecture should be driven by what the problem actually requires, not by the availability of the capability.