# Hyperparameters

Hyperparameters are critical settings or configurations that govern the training process of the model but are not directly learned from the data. 

Unlike model parameters, which are learned automatically during training (e.g., weights and biases), hyperparameters must be set prior to training and can significantly influence the model's performance, efficiency, and ability to generalize to new tasks.

In the fine-tuning phase, hyperparameters play a pivotal role in adapting a pre-trained model to a specific task without extensive retraining from scratch. 

This includes settings such as the <mark style="color:blue;">learning rate</mark>, which determines the size of the steps the model takes during optimisation; the <mark style="color:blue;">batch size</mark>, which affects the amount of data processed simultaneously and influences training stability and speed; and the <mark style="color:blue;">number of epochs</mark>, defining how many times the entire dataset is passed through the model.

#### <mark style="color:green;">Selecting the right set of hyperparameters is crucial</mark>

A learning rate too high might cause the model to overshoot the optimal solution, while one too low may result in a painfully slow convergence. Similarly, an excessively large batch size could lead to poor generalization, and too few epochs might underfit the model to the training data.

The process of hyperparameter tuning involves experimenting with different combinations of hyperparameters to find the set that yields the best performance on a validation dataset.&#x20;

In summary, hyperparameters are the knobs and dials of fine-tuning LLMs, offering a way to customise the training process to achieve optimal performance for specific tasks.&#x20;

Proper tuning of these hyperparameters is essential for unleashing the full potential of LLMs, enabling them to adapt and excel in a wide range of applications.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://training.continuumlabs.ai/training/the-fine-tuning-process/hyperparameters.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.