Vertex AI context caching reduces cost and latency when Gemini requests repeatedly include the same large context. Current Google Cloud documentation distinguishes implicit caching, which happens automatically for supported models when requests share a reusable prefix, from explicit caching, where the application creates a cached-content resource and references it in later requests. The capability is now documented under Gemini Enterprise Agent Platform, but it remains part of the Vertex AI/Gemini production model.
Within AI on Google Cloud, caching is useful for repeated system context such as large policy documents, manuals, long video/audio inputs, codebases, or other shared content sent across many prompts.
The goal is to avoid retransmitting/charging full-price for identical large prefixes while keeping cache lifetime, location, privacy, and invalidation explicit.
Implicit caching is automatic and opportunistic
For supported Gemini models, implicit caching is enabled by default and the service can reuse repeated input prefixes when a cache hit occurs.
Applications do not create or name a resource.
Cache hits are not guaranteed, so design for correct behavior even when every request is processed as uncached input.
Explicit caching gives the application a named resource
An explicit context cache stores selected text, blobs, or supported media under a cached-content resource.
Later Gemini requests reference that resource by name instead of resending the cached content.
This provides predictable reuse for workloads with a stable large context and many follow-up prompts.
The default explicit cache lifetime is 60 minutes
Google currently documents a default expiration of 60 minutes for explicit context caches.
The application can set a different TTL or explicit expiration time and can extend an unexpired cache.
Choose TTL based on how long the source content is valid and how frequently it is reused; a long-lived cache of rapidly changing policy data can reduce cost while increasing staleness risk.
Large cache payloads should live in Cloud Storage
Explicit caches can contain text or supported multimodal content. Google documents that content larger than 10 MB should be referenced through Cloud Storage rather than embedded directly in the create request.
Keep bucket IAM and data location consistent with the cache’s security model.
The cache is not a replacement for source storage; it is a derived serving artifact with a finite lifetime.
Caches are regional resources
Cached content is stored in the region where the cache is created, and regional/model support varies.
The global endpoint is supported for context caching, but Google notes that some features such as CMEK are not supported when using the global endpoint.
Compliance architecture should therefore select regional versus global endpoints deliberately rather than treating cache location as an implementation detail.
CMEK can protect explicit cached content in supported regions
Google supports customer-managed encryption keys for context caching in supported configurations.
Key location, permissions, rotation, and deletion become dependencies for cache availability.
Use CMEK when the application’s security policy requires customer-controlled encryption, but also minimize cached sensitive content because encryption does not remove retention/privacy obligations.
Cached token count is visible in usage metadata
Responses expose cachedContentTokenCount so applications can see how many input tokens were served from cache.
Track cache-hit rate, cached tokens, latency, and cost savings by workload.
A cache that is almost never hit adds management complexity and storage without delivering the intended economic benefit.
Explicit caching discounts are substantial for supported Gemini generations
Current Google guidance states that explicit cache references on Gemini 2.5 and later models receive a large input-token discount compared with uncached input.
Provisioned Throughput also reflects cached tokens with reduced burndown rates for supported models.
Evaluate both cost and latency rather than using caching solely because the discount looks attractive.
Fine-tuned model support can differ
Current Google documentation notes that some fine-tuned Gemini models support implicit caching but not explicit caching.
Do not build one generic cache abstraction and assume every base/tuned model accepts the same cache APIs.
Capability checks should be part of model deployment/migration tests.
Invalidation should follow the source content lifecycle
If the underlying policy, manual, code, or reference media changes, create a new cache and stop referencing the old one.
Use source hashes/version IDs in the cache display metadata or application mapping so a request cannot accidentally use a stale cache after deployment.
Deleting or expiring the cache should be part of the content-release process.
Context caching succeeds when reused context is stable enough to deserve reuse
The mature application chooses implicit versus explicit caching intentionally, tracks hit rate, sets TTL from data freshness, versions source-to-cache mappings, protects region/CMEK policy, and tolerates misses.
Caching is an optimization layer. Correctness must never depend on the assumption that a previous prompt prefix remains cached forever.
Cache candidates should be chosen from repeated context, not merely large context. A 100,000-token document used once gains little from explicit caching, while a 20,000-token policy referenced thousands of times can produce strong savings. Estimate reuse count, token size, TTL, and update frequency before adding cache lifecycle complexity.
Prompt layout affects implicit caching. Keep the large stable prefix—system instructions, shared documents, tool descriptions—at the beginning in consistent form and place user-specific content afterward. Minor differences in the shared prefix can reduce hit probability even when the human-visible meaning is the same.
Explicit cache keys should map to source version hashes. Store a table such as policy_version -> cachedContent resource, and create a new cache when the source changes. Do not reuse a cache resource simply because the filename stayed the same; correctness should follow source content, not path identity.
Cache creation itself has latency and cost. For a new document uploaded by one user, it may be cheaper to send it directly until reuse is proven. For scheduled common context, pre-create the cache before peak traffic so the first user request does not pay creation latency.
TTL extension should be intentional. Automatically extending every cache on each request can make stale content effectively permanent. Only renew while the source version is still active and the expected future reuse justifies it. Otherwise let the cache expire and rebuild from the current source on demand.
Implicit and explicit caching can interact. Google notes that explicit cache usage can coexist with implicit caching of additional repeated prompt content. Metrics should distinguish explicit cached tokens from other input and avoid assuming every discounted token belongs to the resource the application created.
Privacy reviews should treat the cache as another copy of source data. Cached content can contain long documents, audio, video, or code and persists for its TTL. Apply IAM, region selection, CMEK where required, and deletion when the source is revoked. ‘Temporary’ can still be too long for data that must be removed immediately.
Cache errors need a fallback path. If a cached resource expires or is deleted, the application can recreate it or send the content directly depending on latency/cost policy. A user request should not fail permanently simply because an optimization artifact no longer exists.
Model migrations need new cache validation. Explicit caches are tied to compatible model/resource behavior; a newer model may have different support or pricing. Recreate/evaluate caches with the target model instead of assuming the old cache contract can be carried across a migration unchanged.
Cache lifecycle should be integrated into deployment automation. When a new policy bundle or product manual is released, the pipeline can create the new explicit cache, validate one or more requests against it, switch application references, and then let the old cache expire or delete it. This avoids a window where users see mixed versions of the shared context.
Explicit cache use should be visible in tracing. Record cache resource name or source version, cached token count, hit-related metadata, and fallback behavior without logging the cached sensitive content itself. When latency/cost changes, operators can then distinguish a model change from a cache miss or expired resource.
Cache economics should include storage and management overhead as well as token discounts. Very long TTLs can accumulate many obsolete cache versions if deployments create new resources without cleanup. Periodic inventory should remove expired/unused caches and verify active caches still map to current source versions.
Explicit cache creation should be rate-limited and deduplicated. If hundreds of workers independently detect the same missing cache and create identical resources, the application wastes time and can create a cleanup problem. Use a distributed lock or source-version registry so one cache resource is created per shared context/version and other workers reuse it.
Context caching should not be used to smuggle mutable user state into a supposedly static prefix. Per-user account status, permissions, or frequently changing preferences can become stale and be reused incorrectly. Cache shared, stable content; retrieve live transactional or authorization data fresh at the point it is needed.
Cache inventory should be reviewed after model or source decommissioning. Explicit caches linked to retired model versions, deleted source documents, or old application releases should be removed even if they have a long TTL. Stale cache resources add cost, privacy surface, and confusion during troubleshooting when traces reference versions nobody expects to exist.
