使用PEFT庫將原始模型與LoRA權重合并

步驟如下：

• 基礎模型加載：需保持與LoRA訓練時相同的模型配置
• merge_and_unload()：該方法會執行權重合并并移除LoRA層
• 保存格式：合并后的模型保存為標準HuggingFace格式，可直接用于推理
• 代碼如下：

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizerdef merge_and_save_model(base_model_path, lora_path, output_path):# 1. 加載基礎模型和tokenizerprint(f"Loading base model from {base_model_path}")base_model = AutoModelForCausalLM.from_pretrained(base_model_path,torch_dtype=torch.bfloat16,device_map="auto",trust_remote_code=True)tokenizer = AutoTokenizer.from_pretrained(base_model_path,trust_remote_code=True)# 2. 加載LoRA適配器print(f"Loading LoRA adapter from {lora_path}")lora_model = PeftModel.from_pretrained(base_model,lora_path,torch_dtype=torch.float16)# 3. 合并權重并卸載適配器print("Merging LoRA weights with base model")merged_model = lora_model.merge_and_unload()# 4. 保存合并后的模型print(f"Saving merged model to {output_path}")merged_model.save_pretrained(output_path)tokenizer.save_pretrained(output_path)print("Merge completed successfully!")# 使用示例
if __name__ == "__main__":merge_and_save_model(base_model_path= 基礎模型路徑lora_path= lora訓練后的權重output_path= 合并后模型的輸出路徑)