15-20% of Your LLM Requests Are Duplicates. You're Paying for All of Them.

The Pattern That Wastes 15-20%

Duplicate requests show up in predictable places: a user refreshing a page that re-triggers a summarization call, a retry after a flaky network response, multiple users asking the same FAQ-style question, or a background job re-processing the same record because a previous run wasn't marked complete. None of these need a fresh model call — the answer was already computed, sometimes seconds earlier. Without a caching layer, every one of them goes straight back to the model and gets billed again.

Interactive: Caching ROI Calculator

Three Cache Levels

Not every duplicate is the same kind of duplicate, and they need different caching strategies. A deterministic cache handles requests where the exact same input should always produce the exact same output — classification, lookups, structured extraction with temperature 0. A semantic cache handles requests that are worded differently but mean the same thing, using embedding similarity to match against recent answers. A session-scoped cache plus a longer-lived persistent cache handle conversational context — the same user asking a related follow-up, or the same document being summarized again days later.

Interactive: Cache Strategy Decision Tree

The Visibility Problem

You can't fix duplicate waste you can't see. Most teams only discover their duplicate rate after instrumenting every request with enough metadata to compare them — input hash, feature name, user or session ID, and timestamp. Per-request tracking turns "we probably have some duplicate calls" into "feature X has a 22% duplicate rate and costs $1,400/month in repeated calls," which is the difference between a guess and a fix.

Copy// Hash the normalized input and check cache before calling the model
const key = hashInput(normalizedPrompt)
const cached = await cache.get(key)
if (cached) return cached

const response = await callModel(normalizedPrompt)
await cache.set(key, response, { ttlSeconds: 600 })

fetch('https://llm-track.com/api/ingest', {
  method: 'POST',
  headers: { 'x-api-key': process.env.LLMTRACK_KEY },
  body: JSON.stringify({
    provider: 'openai',
    model: response.model,
    feature: 'summarizer',
    total_tokens: response.usage.total_tokens,
    latency_ms: Date.now() - startedAt,
    status: 'success',
    cache_hit: false
  })
}).catch(() => {})

FAQ