改用支持圖片的 AI 模型
qwen-turbo 僅支持文字，要想體驗圖片聊天，需改用 qwen-vl-plus
src/initData.ts
{id: 2,name: "aliyun",title: "阿里 -- 通義千問",desc: "阿里百煉 -- 通義千問",// https://help.aliyun.com/zh/dashscope/developer-reference/api-details?spm=a2c4g.11186623.0.0.5bf41507xgULX5#b148acc634pfcmodels: ["qwen-turbo", "qwen-vl-plus"],avatar:"https://qph.cf2.poecdn.net/main-thumb-pb-4160791-200-qlqunomdvkyitpedtghnhsgjlutapgfl.jpeg",},
安裝依賴 mime-types
用于便捷獲取圖片的類型
npm i mime-types @types/mime-types --save-dev
提問框中選擇本地圖片
src/components/MessageInput.vue
<template><divclass="message-input w-full shadow-sm border rounded-lg border-gray-300 py-1 px-2 focus-within:border-green-700"><div v-if="imagePreview" class="my-2 relative inline-block"><img:src="imagePreview"alt="Preview"class="h-24 w-24 object-cover rounded"/><Iconicon="lets-icons:dell-fill"width="24"@click="delImg"class="absolute top-[-10px] right-[-10px] p-1 rounded-full cursor-pointer"/></div><div class="flex items-center"><inputtype="file"accept="image/*"ref="fileInput"class="hidden"@change="handleImageUpload"/><Iconicon="radix-icons:image"width="24"height="24":class="['mr-2',disabled? 'text-gray-300 cursor-not-allowed': 'text-gray-400 cursor-pointer hover:text-gray-600',]"@click="triggerFileInput"/><inputclass="outline-none border-0 flex-1 bg-white focus:ring-0"type="text"ref="ref_input"v-model="model":disabled="disabled":placeholder="tip"@keydown.enter="onCreate"/><Buttonicon-name="radix-icons:paper-plane"@click="onCreate":disabled="disabled">發送</Button></div></div>
</template><script lang="ts" setup>
import { ref } from "vue";
import { Icon } from "@iconify/vue";import Button from "./Button.vue";const props = defineProps<{disabled?: boolean;
}>();
const emit = defineEmits<{create: [value: string, imagePath?: string];
}>();
const model = defineModel<string>();
const fileInput = ref<HTMLInputElement | null>(null);
const imagePreview = ref("");
const triggerFileInput = () => {if (!props.disabled) {fileInput.value?.click();}
};
const tip = ref("");
let selectedImage: File | null = null;
const handleImageUpload = (event: Event) => {const target = event.target as HTMLInputElement;if (target.files && target.files.length > 0) {selectedImage = target.files[0];const reader = new FileReader();reader.onload = (e) => {imagePreview.value = e.target?.result as string;};reader.readAsDataURL(selectedImage);}
};
const onCreate = async () => {if (model.value && model.value.trim() !== "") {if (selectedImage) {const filePath = window.electronAPI.getFilePath(selectedImage);emit("create", model.value, filePath);} else {emit("create", model.value);}selectedImage = null;imagePreview.value = "";} else {tip.value = "請輸入問題";}
};
const ref_input = ref<HTMLInputElement | null>(null);const delImg = () => {selectedImage = null;imagePreview.value = "";
};defineExpose({ref_input: ref_input,
});
</script><style scoped>
input::placeholder {color: red;
}
</style>
src/preload.ts
需借助 webUtils 從 File 對象中獲取文件路徑
import { ipcRenderer, contextBridge, webUtils } from "electron";
getFilePath: (file: File) => webUtils.getPathForFile(file),
將選擇的圖片，轉存到應用的用戶目錄
圖片很占空間，轉為字符串直接存入數據庫壓力過大，合理的方案是存到應用本地
src/views/Home.vue
在創建會話時執行
const createConversation = async (question: string, imagePath?: string) => {const [AI_providerName, AI_modelName] = currentProvider.value.split("/");let copiedImagePath: string | undefined;if (imagePath) {try {copiedImagePath = await window.electronAPI.copyImageToUserDir(imagePath);} catch (error) {console.error("拷貝圖片失敗:", error);}}// 用 dayjs 得到格式化的當前時間字符串const currentTime = dayjs().format("YYYY-MM-DD HH:mm:ss");// pinia 中新建會話，得到新的會話idconst conversationId = await conversationStore.createConversation({title: question,AI_providerName,AI_modelName,createdAt: currentTime,updatedAt: currentTime,msgList: [{type: "question",content: question,// 如果有圖片路徑，則將其添加到消息中...(copiedImagePath && { imagePath: copiedImagePath }),createdAt: currentTime,updatedAt: currentTime,},{type: "answer",content: "",status: "loading",createdAt: currentTime,updatedAt: currentTime,},],});// 更新當前選中的會話conversationStore.selectedId = conversationId;// 右側界面--跳轉到會話頁面 -- 帶參數 init 為新創建的會話的第一條消息idrouter.push(`/conversation/${conversationId}?type=new`);
};
src/preload.ts
// 拷貝圖片到本地用戶目錄copyImageToUserDir: (sourcePath: string) =>ipcRenderer.invoke("copy-image-to-user-dir", sourcePath),
src/ipc.ts
// 拷貝圖片到本地用戶目錄ipcMain.handle("copy-image-to-user-dir",async (event, sourcePath: string) => {const userDataPath = app.getPath("userData");const imagesDir = path.join(userDataPath, "images");await fs.mkdir(imagesDir, { recursive: true });const fileName = path.basename(sourcePath);const destPath = path.join(imagesDir, fileName);await fs.copyFile(sourcePath, destPath);return destPath;});
將圖片信息傳給 AI
src/views/Conversation.vue
發起 AI 聊天傳圖片參數
// 訪問 AI 模型，獲取答案
const get_AI_answer = async (answerIndex: number) => {await window.electronAPI.startChat({messageId: answerIndex,providerName: convsersation.value!.AI_providerName,selectedModel: convsersation.value!.AI_modelName,// 發給AI模型的消息需移除最后一條加載狀態的消息,使最后一條消息為用戶的提問messages: convsersation.value!.msgList.map((message) => ({role: message.type === "question" ? "user" : "assistant",content: message.content,// 若有圖片信息，則將其添加到消息中...(message.imagePath && { imagePath: message.imagePath }),})).slice(0, -1),});
};
繼續向 AI 提問時圖片參數
const sendNewMessage = async (question: string, imagePath?: string) => {let copiedImagePath: string | undefined;if (imagePath) {try {copiedImagePath = await window.electronAPI.copyImageToUserDir(imagePath);} catch (error) {console.error("拷貝圖片失敗:", error);}}// 獲取格式化的當前時間let currentTime = dayjs().format("YYYY-MM-DD HH:mm:ss");// 向消息列表中追加新的問題convsersation.value!.msgList.push({type: "question",content: question,...(copiedImagePath && { imagePath: copiedImagePath }),createdAt: currentTime,updatedAt: currentTime,});// 向消息列表中追加 loading 狀態的回答let new_msgList_length = convsersation.value!.msgList.push({type: "answer",content: "",createdAt: currentTime,updatedAt: currentTime,status: "loading",});// 消息列表的最后一條消息為 loading 狀態的回答，其id為消息列表的長度 - 1let loading_msg_id = new_msgList_length - 1;// 訪問 AI 模型獲取答案，參數為 loading 狀態的消息的idget_AI_answer(loading_msg_id);// 清空問題輸入框inputValue.value = "";await messageScrollToBottom();// 發送問題后，問題輸入框自動聚焦if (dom_MessageInput.value) {dom_MessageInput.value.ref_input.focus();}
};
src/providers/OpenAIProvider.ts
將消息轉換為 AI 模型需要的格式后傳給 AI
import OpenAI from "openai";
import { convertMessages } from "../util";interface ChatMessageProps {role: string;content: string;imagePath?: string;
}interface UniversalChunkProps {is_end: boolean;result: string;
}export class OpenAIProvider {private client: OpenAI;constructor(apiKey: string, baseURL: string) {this.client = new OpenAI({apiKey,baseURL,});}async chat(messages: ChatMessageProps[], model: string) {// 將消息轉換為AI模型需要的格式const convertedMessages = await convertMessages(messages);const stream = await this.client.chat.completions.create({model,messages: convertedMessages as any,stream: true,});const self = this;return {async *[Symbol.asyncIterator]() {for await (const chunk of stream) {yield self.transformResponse(chunk);}},};}protected transformResponse(chunk: OpenAI.Chat.Completions.ChatCompletionChunk): UniversalChunkProps {const choice = chunk.choices[0];return {is_end: choice.finish_reason === "stop",result: choice.delta.content || "",};}
}
src/util.ts
函數封裝 – 將消息轉換為 AI 模型需要的格式
import fs from 'fs/promises'
import { lookup } from 'mime-types'
export async function convertMessages( messages: { role: string; content: string, imagePath?: string}[]) {const convertedMessages = []for (const message of messages) {let convertedContent: string | any[]if (message.imagePath) {const imageBuffer = await fs.readFile(message.imagePath)const base64Image = imageBuffer.toString('base64')const mimeType = lookup(message.imagePath)convertedContent = [{type: "text",text: message.content || ""},{type: 'image_url',image_url: {url: `data:${mimeType};base64,${base64Image}`}}]} else {convertedContent = message.content}const { imagePath, ...messageWithoutImagePath } = messageconvertedMessages.push({...messageWithoutImagePath,content: convertedContent})}return convertedMessages
}
加載消息記錄中的圖片
渲染進程中，無法直接讀取本地圖片，需借助 protocol 實現
src/main.ts
import { app, BrowserWindow, protocol, net } from "electron";
import { pathToFileURL } from "node:url";
import path from "node:path";// windows 操作系統必要
protocol.registerSchemesAsPrivileged([{scheme: "safe-file",privileges: {standard: true,secure: true,supportFetchAPI: true,},},
]);
在 createWindow 方法內執行
protocol.handle("safe-file", async (request) => {const userDataPath = app.getPath("userData");const imageDir = path.join(userDataPath, "images");// 去除協議頭 safe-file://，解碼 URL 中的路徑const filePath = path.join(decodeURIComponent(request.url.slice("safe-file:/".length)));const filename = path.basename(filePath);const fileAddr = path.join(imageDir, filename);// 轉換為 file:// URLconst newFilePath = pathToFileURL(fileAddr).toString();// 使用 net.fetch 加載本地文件return net.fetch(newFilePath);});
頁面中渲染圖片
src/components/MessageList.vue
img 的 src 添加了 safe-file:// 協議
<div v-if="message.type === 'question'"><div class="mb-3 flex justify-end"><imgv-if="message.imagePath":src="`safe-file://${message.imagePath}`"alt="提問的配圖"class="h-24 w-24 object-cover rounded"/></div><divclass="message-question bg-green-700 text-white p-2 rounded-md">{{ message.content }}</div></div>
最終效果
){kind=icon}
ENFJ人格)
)
