Token Budgets for Multi-Document Prompts

When a prompt contains instructions plus several reference documents plus a question, the order and the share of the budget each piece gets is the most important decision you will make. Most prompts get this wrong.

The 100% budget

• System instructions: 5-10%.
• Reference documents: 50-70%.
• Few-shot examples: 10-20%.
• Conversation history: 5-15%.
• The actual question: 5%.

Why these numbers

Models are pattern matchers. The dominant signal in the context is what they will pattern-match against. If the dominant signal is reference data, you get grounded answers. If it is the question, you get chatty answers. If it is conversation history, you get sycophantic answers.

A worked example

For a 4000-token context: 200 instructions, 2400 reference, 600 examples, 400 history, 200 question. Sum it. Trim the reference before the examples. The model will not miss the examples; it will miss the reference.

Tools

Tokenizers are fast and free. Count everything before you ship. The cheapest way to improve quality is to delete the lowest-value tokens from the largest piece of the budget.