> For the complete documentation index, see [llms.txt](https://training.continuumlabs.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://training.continuumlabs.ai/inference.md).

# INFERENCE

- [Why is inference important?](https://training.continuumlabs.ai/inference/why-is-inference-important.md): Speed and cost counts
- [Grouped Query Attention](https://training.continuumlabs.ai/inference/why-is-inference-important/grouped-query-attention.md)
- [Key Value Cache](https://training.continuumlabs.ai/inference/why-is-inference-important/key-value-cache.md): Managing model memory usage
- [Flash Attention](https://training.continuumlabs.ai/inference/why-is-inference-important/flash-attention.md)
- [Flash Attention 2](https://training.continuumlabs.ai/inference/why-is-inference-important/flash-attention-2.md): The seminal July 2023 paper
- [StreamingLLM](https://training.continuumlabs.ai/inference/why-is-inference-important/streamingllm.md)
- [Paged Attention and vLLM](https://training.continuumlabs.ai/inference/why-is-inference-important/paged-attention-and-vllm.md)
- [TensorRT-LLM](https://training.continuumlabs.ai/inference/why-is-inference-important/tensorrt-llm.md)
- [Torchscript](https://training.continuumlabs.ai/inference/why-is-inference-important/torchscript.md)
- [NVIDIA L40S GPU](https://training.continuumlabs.ai/inference/why-is-inference-important/nvidia-l40s-gpu.md): Low cost inference
- [Triton Inference Server - Introduction](https://training.continuumlabs.ai/inference/why-is-inference-important/triton-inference-server-introduction.md)
- [Triton Inference Server](https://training.continuumlabs.ai/inference/why-is-inference-important/triton-inference-server.md)
- [FiDO: Fusion-in-Decoder optimised for stronger performance and faster inference](https://training.continuumlabs.ai/inference/why-is-inference-important/fido-fusion-in-decoder-optimised-for-stronger-performance-and-faster-inference.md): Google Research, December 2022
- [Is PUE a useful measure of data centre performance?](https://training.continuumlabs.ai/inference/why-is-inference-important/is-pue-a-useful-measure-of-data-centre-performance.md)
- [SLORA](https://training.continuumlabs.ai/inference/why-is-inference-important/slora.md)