The fine-tuning technique of large language models is a key method for adapting general models to specific tasks. As the model size grows from billions to hundreds of billions of parameters, traditional full parameter fine-tuning becomes increasingly expensive, giving rise to various efficient fine-tuning schemes.
Full parameter fine-tuning is the most traditional way to train all parameters of a model using specific task data. The effect is usually the best, but the cost is also the highest. Taking Llama 3 70B as an example, full parameter fine-tuning requires at least 8 A100 GPUs and several days of training to complete. This approach is suitable for scenarios that require extremely high model quality.
LoRA (Low Rank Adaptation) is currently the most popular parameter efficient fine-tuning method. Its core is to keep the original model weights unchanged and inject only a small number of trainable low rank matrices in specific layers. The parameter count is usually only 0.1% to 1% of the original model, and can be fine tuned on a single consumer grade graphics card.
QLoRA introduces 4-bit quantization on top of LoRA, reducing video memory requirements by approximately 4 times. A RTX 4090 with 24GB can run, and the fine-tuning effect is only 1-2 percentage points lower than LoRA, significantly reducing the hardware threshold.
Preface fine-tuning and prompt word fine-tuning are another lightweight approach that freezes the entire model and only learns a small number of vector embeddings, suitable for scenarios that do not require high task specificity. Adapter fine-tuning involves inserting small neural network modules between each layer of the Transformer, achieving a good balance between parameter quantity and performance.
For teams with sufficient hardware, full parameter fine-tuning is still the most effective choice. For most scenarios, LoRA offers the highest cost-effectiveness. QLoRA is suitable for individual developers who have limited resources but want to try fine-tuning. In practical applications, strategies need to be selected comprehensively based on task complexity, data size, and hardware conditions.