參考鏈接
知乎帖子
B站視頻
huggingface 鏡像網站（不太全，比如 Qwen/Qwen2.5-VL-7B-Instruct就沒有）
huggingface 5種下載方式匯總
通過huggingface-cli下載模型
不一樣的部分是預訓練權重的下載和demo
首先安裝huggingface_hub
pip install -U huggingface_hub
設置鏡像
export HF_ENDPOINT=https://hf-mirror.com
windows端需要添加系統變量。
名稱：HF_ENDPOINT，值： "https://hf-mirror.com"
然后通過huggingface-cli下載模型，
huggingface-cli download --resume-download Qwen/Qwen2.5-VL-7B-Instruct --local-dir ./ --local-dir-use-symlinks False --resume-download
參考：通過huggingface-cli下載模型
運行DEMO
加載模型方式
如果希望下載到指定的目錄，可以往from_pretrained方法傳入cache_dir 參數，如下所示：
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("internlm/internlm2-chat-7b", torch_dtype=torch.float16, trust_remote_code=True, cache_dir='/home/{username}/huggingface').cuda()
運行以下代碼
import gradio as gr
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import torch# 加載模型和處理器
model = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")def process_image_and_text(image, text_prompt):if image is None:return "請上傳一張圖片。"# 構建消息格式messages = [{"role": "user","content": [{"type": "image","image": image, # Gradio將自動處理圖片路徑},{"type": "text", "text": text_prompt if text_prompt else "Describe this image."},],}]try:# 準備推理輸入text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)image_inputs, video_inputs = process_vision_info(messages)inputs = processor(text=[text],images=image_inputs,videos=video_inputs,padding=True,return_tensors="pt",)inputs = inputs.to(model.device)# 生成輸出with torch.no_grad():generated_ids = model.generate(**inputs, max_new_tokens=128)generated_ids_trimmed = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]output_text = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)return output_text[0]except Exception as e:return f"處理過程中出現錯誤: {str(e)}"# 創建Gradio界面
with gr.Blocks() as demo:gr.Markdown("# Qwen2.5-VL 圖像理解演示")with gr.Row():with gr.Column():image_input = gr.Image(type="filepath", label="上傳圖片")text_input = gr.Textbox(placeholder="請輸入提示語（如不輸入，默認描述圖片）", label="提示語")submit_btn = gr.Button("提交")with gr.Column():output = gr.Textbox(label="輸出結果")submit_btn.click(fn=process_image_and_text,inputs=[image_input, text_input],outputs=output)gr.Examples(examples=[["path/to/example1.jpg", "這張圖片里有什么？"],["path/to/example2.jpg", "描述圖中的場景"],],inputs=[image_input, text_input],)# 啟動應用
if __name__ == "__main__":demo.launch(share=True)
)
