LLM API 'prompt is too long' / context length exceeded errors: how do I trim, summarize or compact conversation history, and why does max_tokens matter?

Cause: Everything in a request counts toward the model's context window: the system prompt, every message including tool results, images and documents, tool definitions, and the output the model generates (including thinking). Claude's docs state that if the input alone already exceeds the window, the API returns a 400 invalid_request_error ("prompt is too long"). On Claude 4.5 models and newer, input plus max_tokens exceeding the window is accepted and generation simply stops with stop_reason "model_context_window_exceeded"; earlier models return a validation error instead. Other providers have similar errors, so check your provider's current docs for the model's window. Fix (best first): 1. Measure before sending. Use the provider's token counting endpoint (Claude has a token counting API) and compare against the model's window; keep headroom for max_tokens. 2. Use server-side compaction where available. Claude's docs call it the primary strategy for long-running conversations: earlier turns are summarized on the server so the conversation can continue (beta for Claude 4.6 and later models; check the compaction page for current status). 3. Use context editing for specific bloat: clear old tool results in agent loops and clear old thinking blocks. 4. In your own code, keep the system prompt and the most recent turns verbatim, replace older turns with a short running summary (generated by a cheaper model), and truncate or retrieve (RAG) large documents instead of inlining them. Never drop a tool_use without its matching tool_result, or the reverse. 5. Cut fixed overhead: trim tool definitions or defer them with a tool-search mechanism, and avoid resending large images each turn. 6. Handle truncation: if stop_reason is max_tokens or model_context_window_exceeded, treat the output as incomplete rather than final. Verify: Token count for the next request stays below the window with room for output, and long sessions no longer return the context error. Sources: https://platform.claude.com/docs/en/build-with-claude/context-windows https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons