> For the complete documentation index, see [llms.txt](https://training.continuumlabs.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://training.continuumlabs.ai/knowledge/retrieval-augmented-generation/optimizing-instructions-and-demonstrations-for-multi-stage-language-model-programs.md).

# Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs

This paper addresses the challenge of optimizing prompts for multi-stage language model (LM) programs, which are sophisticated pipelines composed of modular LM calls. 

The goal is to maximise a downstream metric without requiring access to module-level labels or gradients. The authors propose a novel optimizer called MIPRO, which employs several strategies to optimize free-form instructions and few-shot demonstrations for each module within the pipeline.

{% embed url="<https://arxiv.org/abs/2406.11695>" %}