Fine-tuning is the process of further training an already pretrained model on a smaller, specialized dataset to adapt it to a specific task or domain. Whereas pretraining is largely self-supervised and imparts broad general knowledge, fine-tuning is typically a supervised process using labeled examples.
During fine-tuning, the model receives input prompts from the training data, generates outputs, compares them with the expected answers and adjusts its internal weights based on the error. This turns a general-purpose base model into a specialist — for example an assistant tuned for legal questions or a model for medical text classification.
Besides full fine-tuning, resource-efficient methods have become established (parameter-efficient fine-tuning such as LoRA), which adjust only a small fraction of the weights. Fine-tuning is an alternative or complement to prompt engineering and RAG when a model needs to follow a particular style, format or domain consistently.