Context Window Budget

Every model has a finite context window measured in tokens, and managing it as a budget is essential for effective agent design. You must allocate space for system instructions, conversation history, retrieved context, tool definitions, and leave room for the model's reasoning and output — exceeding the context window causes silent truncation or errors, while wasting tokens on irrelevant information degrades model performance even within the limit. Research on the "lost in the middle" problem shows that models disproportionately attend to information at the beginning and end of the context, making strategic placement of critical information as important as quantity. Understanding context budgeting is foundational to every agentic workflow because it determines how much information the agent can reason over in a single inference step. This concept connects to token economics for understanding the cost side of context usage, context caching for reusing expensive context prefixes, and RAG patterns for dynamically retrieving only the most relevant information.