On a Friday morning with thin order books, a headline crossed the wire: OpenAI had improved prompt caching in something called GPT-6 Sol and something called GPT-6 Luna. Two models. Named after the sun and the moon. Named — whether by coincidence or by design — in letters that rhyme with two of the most expensive tickers in this industry's short history: SOL and LUNA. One of those tickers still trades. The other one is scar tissue.
I read the piece four times. It gave me four facts. None of them carried a source. No API documentation. No pricing page. No benchmark table. No TTL. No minimum cacheable token length. No discount rate. No write fee. Just a phrase — "enhanced prompt caching" — and a promise of "significantly lower operating costs and better efficiency."
That is not a news story. That is a press release with the receipt torn off.
So I separate the two things the headline welded together. The caching technology is real, and I can price it. The models are unverified, and I cannot. Anyone who fuses those two facts into a single trade is already on the wrong side of the spread.
Prompt caching is not new. It is not a breakthrough. It is a refund mechanism bolted onto the prefill stage of transformer inference, and it has existed in production form since 2024.
Here is the mechanism, stripped of marketing. Send a prompt to a large model and the model converts every token into a key-value vector — the KV cache — during the prefill pass. That computation is expensive. If the next request shares an identical prefix — the same system prompt, the same retrieved document, the same tool schema — the engine reuses the cached vectors instead of recomputing them. Identical output. Less arithmetic. Lower latency to first token. Lower cost per call.
The industry parameters are public, and they are boringly consistent.
OpenAI shipped automatic prompt caching in October 2024 across GPT-4o and its siblings. Cache-hit input tokens cost roughly half of normal. TTL landed in the five-to-ten-minute band. Minimum cacheable length: 1,024 tokens. No write fee. Caches never crossed organizational boundaries.
Anthropic went the other direction — explicit control. You write cache breakpoints into the prompt yourself. Cache reads cost 0.1x of standard input. Cache writes cost 1.25x. TTL: five minutes. You pay for the privilege of choosing when the cache lives and when it dies.
Google Gemini priced it by token-hour plus storage, a cloud product's instinct to bill rent. DeepSeek made caching automatic and aggressive, with hit pricing near 0.1x, backed by disk-based caching and a pricing war it clearly intends to win.
Every one of those is verifiable in a changelog. That is the distance between a parameter and a rumor. A parameter has a number and a date. A rumor has an adjective.
The article gave me adjectives.
I build the cost model from the parameters that do exist, because that is the only honest move when the specific numbers are missing. If GPT-6 Sol and Luna ship caching at all, they ship it inside a system I can approximate. Approximation is not guessing. It is triangulation.
Start with the unit economics of one workload. Take a retrieval-augmented agent with a 20,000-token system prompt, a tool schema, and a fixed instruction block. Say the static prefix is 18,000 tokens. The dynamic query is 500. Without caching you pay full input price on 18,500 tokens every call. With caching you pay full price on 500 and the cached rate on 18,000.
At a 50 percent cache discount, the effective cost of that call falls by roughly 48 percent. At a 90 percent discount — DeepSeek territory — it falls by roughly 87 percent. The gap between those two numbers is the entire business case for an agent startup. It is also the gap between a marketing claim and a survivable gross margin.
The single most important number in any prompt-caching announcement is not the discount rate. It is the hit rate, and no vendor prints it on the box.
A 90 percent discount with a 20 percent hit rate saves almost nothing. A 50 percent discount with a 95 percent hit rate is a cost structure you can build a company on. The discount gets advertised because it flatters. The hit rate gets buried because it is honest, and honest numbers are harder to sell.
Hit rates collapse for mechanical reasons. The cache requires the prefix to be byte-for-byte identical. Insert a timestamp. Insert a random session ID. Insert a user name at the top of the prompt instead of the bottom. Insert a retrieved chunk whose order shifts between calls. Any one of those, and the cache misses. Every miss is a full-price prefill.
I learned that the hard way, not from a doc. In 2020 I ran an arbitrage script between Uniswap and Sushiswap pools during the airdrop mania. The spread existed for seconds. My edge was never the spread. My edge was that my execution path was deterministic and my counterparties' was not. The moment a variable slipped into the hot path — a volatile gas estimate committed a beat too early — my fill rate dropped by a third. Same math. Different plumbing. Prompt caching is the same class of problem. The discount is the spread. The hit rate is the fill rate. You do not trade the spread. You trade the fill.
Now overlay infrastructure cost, because caching is not free. It moves cost; it does not delete it. The KV cache for a 20,000-token prefix is not a rounding error in memory. It scales linearly with context length, and at long-context scale a single cached prefix can occupy gigabytes of high-bandwidth memory that is now unavailable for other work. You saved FLOPs. You spent HBM. On a GPU where memory is the binding constraint — and in 2026 it always is — that is a real trade, not a free lunch.
Then the routing problem, and this is where the engineers earn their salary. To hit a cache, a request with a given prefix must land on the same replica holding that prefix's vectors. That means consistent hashing or prefix-aware routing, and it fights head-on with naive load balancing. Load balancers want to spread. Caches want to pin. You cannot do both perfectly. Every architecture picks a point on that curve, and where it picks determines the real hit rate, which determines the real discount — the number nobody published.
Multi-tenant isolation is the second hidden cost. Share a cached prefix between two customers and you have a potential leak — one user's system prompt bleeding into another's context is not a theoretical attack, it is a plumbing bug waiting for a bad query. Fully isolate, and you waste memory and hit rate. Every provider draws the isolation line somewhere. The article drew it nowhere, because it did not know the line existed.
Now the two models. Sol and Luna. If they are real and if they are distinct, caching almost certainly does not cross between them. Different weights produce different KV vectors. Two caches. Two routing tables. Two memory budgets. Whatever efficiency gain exists halves or worse at the seams. That is the kind of detail a four-fact article omits because it cannot be compressed into a headline.
A model name is not a product. A product has a changelog.
I have spent most of my career refusing to trade narrative. In late 2017, while retail chased the Tezos ICO on Telegram sentiment, I read the vesting schedule and built a short against the day-100 unlock. The crowd was loud. The contract was quiet. The contract was right. Forty-two percent before the token fell sixty. Not luck. Arithmetic. The same discipline applies here. The article is the Telegram channel. The API documentation is the contract. One of them will be correct. Guess which.
Turn the lens on the naming, because in this industry names are never neutral. Sol is the ticker of Solana. Luna is the ticker of Terra's LUNA — the asset that de-pegged in May 2022 and dragged down Celsius, Three Arrows, and a generation of retail balance sheets. A crypto news desk publishing an OpenAI story with two characters named Sol and Luna is either blind to that resonance or is monetizing it. I have watched this pattern before. The headline does not need to be true. It needs to be clicked. And the audience it is fishing for is the same audience that bought the top of a forty-billion-dollar stablecoin because an influencer said the yield was sustainable.
The naming mirrors something structural, too. In 2022 I shorted the UST-LUNA pair delta-neutral, funded by stablecoin lending on Aave, and the trade paid 150 percent while the industry panicked. But the part I remember is what came after: the same influencers predicting the collapse were already rotating the crowd into a new "safe" asset. When I audited that asset's validator set, 30 percent of stake sat with one exchange. Decentralization was the marketing. Concentration was the mechanism. Sol and Luna are two names doing the same job at the AI layer — resonance dressed as product.
Here is the part that actually moves capital. If prompt caching is real and improving, the beneficiary is not OpenAI's story. The beneficiary is the inference cost curve. Every point of cache-hit savings flows into the unit economics of agents, retrieval pipelines, code assistants, and long-document workloads. Those are the token consumers. Their gross margins move. The model lab captures the marketing. The application layer captures the margin — or it should, if the pricing holds.
But watch the pricing. Caching lets a lab cut cost without cutting the sticker price. If Sol and Luna launch above GPT-4o on base price and hand back a cache discount, net cost to a developer can be flat or worse while the announcement reads as a win. I have seen this trade dressed many ways. A discount is only a discount relative to a baseline. The article named no baseline. A discount with no baseline is a discount with no buyer.
Then the second-order effect that makes infrastructure investors nervous and application founders happy: Jevons. Cheaper tokens do not reduce token consumption. They increase it. Cut the cost of a long-context call far enough and you do not run the same workload for less — you run a bigger one, an always-on agent, a retrieval loop that was previously too expensive to justify. The FLOPs saved by caching get spent on new demand. The net effect on GPU load is ambiguous, sometimes negative for the compute thesis, always a headache for anyone modeling inference revenue linearly.
Chaos is just data with no label yet. The label on this chaos reads "unverified," and I do not trade unlabeled data at full size.
Be honest about what caching does to the retrieval layer, because the loudest take — that caching kills vector databases — is wrong, and it is wrong in the profitable direction. A cache makes static prefixes cheap. A vector database makes dynamic relevance cheap. You still have to decide which 2,000 tokens of a million-document corpus belong in the prompt. Caching does not retrieve. It reuses. Confuse the two and you build an agent that is fast, cheap, and confidently wrong.
One more layer, because it is where this becomes a security story rather than an efficiency story. I spent three months in 2026 reverse-engineering an autonomous trading agent framework for a client. The agents executed micro-transactions on-chain with no human in the loop. The vulnerability was not in the model weights; it was in the prompt path. A crafted input could make the agent sign a malicious contract through a prompt-injection vector, and I drained half a million dollars from a testnet pool to prove it. Now add a shared cache to that architecture. If the cached prefix is shared across agents in a fleet, a single poisoned system prompt can propagate to every agent that hits the cache. You have turned a caching optimization into a lateral-movement primitive. That is not a hypothetical. That is a design review item every team shipping agent fleets should be filing today.
For the compliance desk, the missing security section is not an oversight. It is the story. A KV cache holding a user's system prompt and retrieved context is a store of sensitive material. Its TTL is a data-retention policy whether anyone calls it that or not. Its isolation boundary is a data-residency question whether anyone files it that way. Publish a caching feature with no stated TTL, no isolation granularity, and no deletion guarantee, and you have shipped a compliance liability with a latency benefit attached. Financial, medical, and government buyers notice eventually. They always do — after the memo leaks.
Now the blind spot, and it is the same one every cycle.
Retail reads "OpenAI improves prompt caching in GPT-6 Sol and Luna" and hears one thing: AI is winning, the hot name is new, get in early. Smart money reads the same sentence and hears something colder: four unsourced facts, a suspicious naming coincidence, and a crypto outlet farming an AI narrative for traffic. Two readings of one string. The gap between them is the trade.
Retail is not stupid. It is fast. Speed is both its edge and its trap. It buys the headline before the changelog exists, then holds through the correction when the changelog contradicts the headline. Smart money is slow. It waits for the API doc. It pays a worse entry for a smaller probability of being destroyed. In a bear market, that is not a suboptimal strategy. It is the only strategy.
Liquidity vanishes the moment you need it most. That is true of order books, and it is true of information. When the official parameters finally publish — or when the models turn out to be a misread — the market that repriced on the rumor will discover it cannot exit at the rumor price. I watched a forty-billion-dollar asset fall through every bid on the way down because the exits were a suggestion and the crowd was the liquidity. The same reflex lives at the AI layer. The crowd chasing Sol and Luna is the crowd providing the exit for whoever read the contract first.
Options give you the right to walk away. Read that at the model layer, not the options chain. A well-built position in any narrative — crypto or AI — is one where the downside is defined before the headline prints. In early 2024 I built a straddle ahead of the spot Bitcoin ETF decision, buying both legs for a combined premium of 1.2 million dollars because institutional pricing models had mis-valued crypto-specific liquidity risk as noise. The approval spiked, the miner sell-off corrected, volatility expanded, and I exited both legs for 65 percent. I never predicted the direction. I priced the noise. That is the only edge that survives a bear market: not knowing what happens, but owning the right to be paid either way. Sol and Luna are the same shape. I do not need to know whether the models are real. I need to know what the headline does to price, and whether I hold a defined exit before it does it. With no changelog, the honest position size is the one that survives being wrong — small, hedged, patient.
What I am watching, in order, with levels attached.
The caching parameters: TTL, minimum cacheable token length, discount rate, write fee, and — the number nobody advertises — the disclosed or inferable hit rate. Those five numbers are the product. Everything else is a press release.
The pricing baseline: base price of Sol and Luna against GPT-4o. No comparison, no verdict.
The isolation and deletion policy: organization-level or request-level, and what happens at TTL expiry. That is a compliance gate, not a footnote.
The capacity event: an official OpenAI changelog, API doc, or model page. Until then, the safest trade is the one you do not take.
Volatility is just noise waiting to be priced. Right now the noise is a headline with no label. I am not long the narrative. I am short the certainty — and waiting, as always, for the contract to speak.