Context Compression
Techniques to fit more useful information into a limited context window.
Context compression is a collection of techniques for maximising the information density of what you put into a model's context window — important when your content exceeds the available space.
Methods:
Summarisation: Have the model (or a cheaper model) compress older conversation turns into a dense summary that replaces the raw history. Reduces tokens dramatically while preserving key facts.
Selective retrieval (RAG): Instead of stuffing all documents into context, embed them and retrieve only the most relevant passages at query time. 100 documents → 3 relevant passages.
Structured extraction: Extract key facts from verbose text (e.g., a 10,000-word earnings call transcript → a 200-token structured JSON of key numbers).
Chunking strategies: For long documents, split smartly at semantic boundaries (paragraphs, sections) rather than fixed character counts — preserves coherence per chunk.
Hypothetical Document Embeddings (HyDE): Generate a hypothetical answer to the query, embed it, and use it to retrieve similar actual documents — often better recall than embedding the raw question.
When it matters most: Long-running agentic workflows where conversation history accumulates, enterprise RAG over large document collections, and any task where input cost dominates (batch document processing).
In plain terms
Packing for a trip: don't take the whole wardrobe, take what you actually need for the weather forecast. The forecast is your query; your packed bag is the compressed, relevant context.