Last month's Anthropic bill hit $312. Sixty percent of it traced back to a single 6-hour window when I was doing an in-place Vectorize index rebuild.
Here's the part I didn't expect: the rebuild didn't throw errors. Queries returned results, scores just quietly dropped from 0.78 to 0.61. My agent interpreted that as "no relevant docs found" and fell back to Claude directly — 2,880 times over six hours. At ~5K input tokens per query on Sonnet, that's $43 in LLM fallback costs from what I thought was a routine maintenance task.
The fix was a Blue/Green swap. I spun up ads-insights-v2 alongside the existing index, rebuilt all 180K vectors into it using 256-token chunks (down from 512), validated against a 20-query golden set, then flipped one environment variable:
wrangler deploy --var VECTORIZE_INDEX_NAME:ads-insights-v2
Actual downtime: 4.19 seconds. The old index kept serving traffic the entire 8.5 hours the new one was being populated. Running two indexes in parallel for roughly a day cost $1.20 in extra Vectorize storage. So the real cost comparison was $43 (in-place) vs $5.50 (Blue/Green). Not a close call.
The root cause was something I'd been misreading for weeks. My RAG recall had been degrading because 512-token chunks were too coarse for ad insight data — shorter chunks improved top-1 similarity scores measurably. But because the degradation was gradual and silent, I kept blaming prompt quality and LLM behavior instead of the index structure. The $312 bill was the thing that finally made me look at the actual confidence scores in my query logs.
One practical note on batch sizing: the Vectorize docs say 1,000 vectors per upsert request, but I hit intermittent timeouts above 500. Kept batches at 500 with retry logic and had zero failures across the full 180K rebuild.
I wrote up the full breakdown — including the step-by-step rebuild procedure, the exact golden-set validation I used before flipping traffic, and where Blue/Green costs would flip negative at larger index sizes — over on riversealab.com.
