一、前言
? ? Qdrant 是一個高性能的向量搜索引擎,廣泛應用于相似性搜索、推薦系統和大規模數據檢索等場景。雖然其原生 API 提供了強大的功能,但對于開發者和運維人員來說,缺乏直觀的可視化界面常常增加了使用門檻。為了解決這一問題,Qdrant Web UI 應運而生——它提供了一個簡潔友好的前端操作界面,幫助用戶更高效地管理集合、查看數據、執行查詢和監控服務狀態。
? ? 本文將手把手帶你搭建 Qdrant Web UI,從環境準備到部署運行,一步步實現對 Qdrant 服務的可視化管理,讓向量數據庫的操作變得更加簡單直觀。
二、術語
2.1、向量數據庫
? ? 向量數據庫是一種專門用于存儲和處理高維向量數據的數據庫系統。與傳統的關系型數據庫或文檔數據庫不同,向量數據庫的設計目標是高效地支持向量數據的索引和相似性搜索。
? ? 在傳統數據庫中,數據通常是以結構化的表格形式存儲,每個記錄都有預定義的字段。但是,對于包含大量高維向量的數據,如圖像、音頻、文本等,傳統的數據庫模型往往無法有效地處理。向量數據庫通過引入特定的數據結構和索引算法,允許高效地存儲和查詢向量數據。
? ? 向量數據庫的核心概念是向量索引。它使用一種稱為向量空間模型的方法,將向量映射到多維空間中的點,并利用這種映射關系構建索引結構。這樣,當需要搜索相似向量時,可以通過計算向量之間的距離或相似度來快速定位相似的向量。
2.2、向量數據庫的使用場景
? ? ?向量數據庫在許多領域中都有廣泛的應用場景,特別是涉及到高維向量數據存儲和相似性搜索的任務。以下是一些常見的使用場景:
- 目標識別和圖像搜索:向量數據庫可用于存儲圖像特征向量,以支持快速的相似圖像搜索和目標識別。它在圖像搜索引擎、人臉識別和視頻監控等領域具有重要作用。
- 推薦系統:向量數據庫可以存儲用戶和物品的特征向量,用于個性化推薦。基于相似性搜索,可以找到與用戶興趣相似的物品,提供個性化的推薦結果。
- 自然語言處理:在文本處理任務中,可以使用向量數據庫存儲文本向量,如詞向量、句向量等。基于相似性搜索,可以進行文本匹配、語義相似度計算等操作。
- 數據聚類和分類:向量數據庫可用于高維向量數據的聚類和分類分析。它可以幫助發現數據集中的聚類模式和類別,用于數據挖掘和機器學習任務。
- 檢索與推薦系統:在電子商務和商品搜索中,向量數據庫可以存儲商品特征向量,以支持相似商品的搜索和推薦。它可以提供更準確和個性化的搜索結果。
- 醫療和生物信息學:向量數據庫可用于存儲基因表達向量、蛋白質特征向量等生物信息學數據。它可以在基因組學、藥物研發等領域中幫助進行數據分析和研究。
- 視頻內容分析:向量數據庫可用于存儲視頻特征向量,如視頻幀特征、視頻片段特征等。它可以用于視頻內容搜索、視頻剪輯和視頻推薦等應用。
2.3、Qdrant
? ? Qdrant 是一個高性能、易用、功能豐富的開源向量搜索引擎,適用于需要處理大量向量數據并執行相似性搜索的各種 AI 應用。它結合了現代索引技術、靈活的數據模型以及強大的 API 接口,非常適合構建智能推薦、圖像檢索、語義搜索等系統。
三、前置條件
3.1、下載Qdrant
https://github.com/qdrant/qdrant/releases/tag/v1.14.1https://github.com/qdrant/qdrant/releases/tag/v1.14.1? ? 此次以Windows系統為例,下載文件:qdrant-x86_64-pc-windows-msvc.zip
? ? 解壓qdrant-x86_64-pc-windows-msvc.zip得到一個qdrant.exe文件
3.2、新建config配置文件
? 在qdrant.exe文件的同級目錄創建一個config文件夾,并在config文件夾里面創建一個config.yaml,并寫入以下內容
log_level: INFO# Logging configuration
# Qdrant logs to stdout. You may configure to also write logs to a file on disk.
# Be aware that this file may grow indefinitely.
# logger:
# # Logging format, supports `text` and `json`
# format: text
# on_disk:
# enabled: true
# log_file: path/to/log/file.log
# log_level: INFO
# # Logging format, supports `text` and `json`
# format: textstorage:# Where to store all the datastorage_path: E:/qdrant/storage# Where to store snapshotssnapshots_path: E:/qdrant/snapshotssnapshots_config:# "local" or "s3" - where to store snapshotssnapshots_storage: local# s3_config:# bucket: ""# region: ""# access_key: ""# secret_key: ""# Where to store temporary files# If null, temporary snapshots are stored in: storage/snapshots_temp/temp_path: null# If true - point payloads will not be stored in memory.# It will be read from the disk every time it is requested.# This setting saves RAM by (slightly) increasing the response time.# Note: those payload values that are involved in filtering and are indexed - remain in RAM.# # Default: trueon_disk_payload: true# Maximum number of concurrent updates to shard replicas# If `null` - maximum concurrency is used.update_concurrency: null# Write-ahead-log related configurationwal:# Size of a single WAL segmentwal_capacity_mb: 32# Number of WAL segments to create ahead of actual data requirementwal_segments_ahead: 0# Normal node - receives all updates and answers all queriesnode_type: "Normal"# Listener node - receives all updates, but does not answer search/read queries# Useful for setting up a dedicated backup node# node_type: "Listener"performance:# Number of parallel threads used for search operations. If 0 - auto selection.max_search_threads: 0# Max number of threads (jobs) for running optimizations across all collections, each thread runs one job.# If 0 - have no limit and choose dynamically to saturate CPU.# Note: each optimization job will also use `max_indexing_threads` threads by itself for index building.max_optimization_threads: 0# CPU budget, how many CPUs (threads) to allocate for an optimization job.# If 0 - auto selection, keep 1 or more CPUs unallocated depending on CPU size# If negative - subtract this number of CPUs from the available CPUs.# If positive - use this exact number of CPUs.optimizer_cpu_budget: 0# Prevent DDoS of too many concurrent updates in distributed mode.# One external update usually triggers multiple internal updates, which breaks internal# timings. For example, the health check timing and consensus timing.# If null - auto selection.update_rate_limit: null# Limit for number of incoming automatic shard transfers per collection on this node, does not affect user-requested transfers.# The same value should be used on all nodes in a cluster.# Default is to allow 1 transfer.# If null - allow unlimited transfers.#incoming_shard_transfers_limit: 1# Limit for number of outgoing automatic shard transfers per collection on this node, does not affect user-requested transfers.# The same value should be used on all nodes in a cluster.# Default is to allow 1 transfer.# If null - allow unlimited transfers.#outgoing_shard_transfers_limit: 1# Enable async scorer which uses io_uring when rescoring.# Only supported on Linux, must be enabled in your kernel.# See: <https://qdrant.tech/articles/io_uring/#and-what-about-qdrant>#async_scorer: falseoptimizers:# The minimal fraction of deleted vectors in a segment, required to perform segment optimizationdeleted_threshold: 0.2# The minimal number of vectors in a segment, required to perform segment optimizationvacuum_min_vector_number: 1000# Target amount of segments optimizer will try to keep.# Real amount of segments may vary depending on multiple parameters:# - Amount of stored points# - Current write RPS## It is recommended to select default number of segments as a factor of the number of search threads,# so that each segment would be handled evenly by one of the threads.# If `default_segment_number = 0`, will be automatically selected by the number of available CPUsdefault_segment_number: 0# Do not create segments larger this size (in KiloBytes).# Large segments might require disproportionately long indexation times,# therefore it makes sense to limit the size of segments.## If indexation speed have more priority for your - make this parameter lower.# If search speed is more important - make this parameter higher.# Note: 1Kb = 1 vector of size 256# If not set, will be automatically selected considering the number of available CPUs.max_segment_size_kb: null# Maximum size (in KiloBytes) of vectors to store in-memory per segment.# Segments larger than this threshold will be stored as read-only memmapped file.# To enable memmap storage, lower the threshold# Note: 1Kb = 1 vector of size 256# To explicitly disable mmap optimization, set to `0`.# If not set, will be disabled by default.memmap_threshold_kb: null# Maximum size (in KiloBytes) of vectors allowed for plain index.# Default value based on https://github.com/google-research/google-research/blob/master/scann/docs/algorithms.md# Note: 1Kb = 1 vector of size 256# To explicitly disable vector indexing, set to `0`.# If not set, the default value will be used.indexing_threshold_kb: 20000# Interval between forced flushes.flush_interval_sec: 5# Max number of threads (jobs) for running optimizations per shard.# Note: each optimization job will also use `max_indexing_threads` threads by itself for index building.# If null - have no limit and choose dynamically to saturate CPU.# If 0 - no optimization threads, optimizations will be disabled.max_optimization_threads: null# This section has the same options as 'optimizers' above. All values specified here will overwrite the collections# optimizers configs regardless of the config above and the options specified at collection creation.#optimizers_overwrite:# deleted_threshold: 0.2# vacuum_min_vector_number: 1000# default_segment_number: 0# max_segment_size_kb: null# memmap_threshold_kb: null# indexing_threshold_kb: 20000# flush_interval_sec: 5# max_optimization_threads: null# Default parameters of HNSW Index. Could be overridden for each collection or named vector individuallyhnsw_index:# Number of edges per node in the index graph. Larger the value - more accurate the search, more space required.m: 16# Number of neighbours to consider during the index building. Larger the value - more accurate the search, more time required to build index.ef_construct: 100# Minimal size (in KiloBytes) of vectors for additional payload-based indexing.# If payload chunk is smaller than `full_scan_threshold_kb` additional indexing won't be used -# in this case full-scan search should be preferred by query planner and additional indexing is not required.# Note: 1Kb = 1 vector of size 256full_scan_threshold_kb: 10000# Number of parallel threads used for background index building.# If 0 - automatically select.# Best to keep between 8 and 16 to prevent likelihood of building broken/inefficient HNSW graphs.# On small CPUs, less threads are used.max_indexing_threads: 0# Store HNSW index on disk. If set to false, index will be stored in RAM. Default: falseon_disk: false# Custom M param for hnsw graph built for payload index. If not set, default M will be used.payload_m: null# Default shard transfer method to use if none is defined.# If null - don't have a shard transfer preference, choose automatically.# If stream_records, snapshot or wal_delta - prefer this specific method.# More info: https://qdrant.tech/documentation/guides/distributed_deployment/#shard-transfer-methodshard_transfer_method: null# Default parameters for collectionscollection:# Number of replicas of each shard that network tries to maintainreplication_factor: 1# How many replicas should apply the operation for us to consider it successfulwrite_consistency_factor: 1# Default parameters for vectors.vectors:# Whether vectors should be stored in memory or on disk.on_disk: null# shard_number_per_node: 1# Default quantization configuration.# More info: https://qdrant.tech/documentation/guides/quantizationquantization: null# Default strict mode parameters for newly created collections.strict_mode:# Whether strict mode is enabled for a collection or not.enabled: false# Max allowed `limit` parameter for all APIs that don't have their own max limit.max_query_limit: null# Max allowed `timeout` parameter.max_timeout: null# Allow usage of unindexed fields in retrieval based (eg. search) filters.unindexed_filtering_retrieve: null# Allow usage of unindexed fields in filtered updates (eg. delete by payload).unindexed_filtering_update: null# Max HNSW value allowed in search parameters.search_max_hnsw_ef: null# Whether exact search is allowed or not.search_allow_exact: null# Max oversampling value allowed in search.search_max_oversampling: nullservice:# Maximum size of POST data in a single request in megabytesmax_request_size_mb: 32# Number of parallel workers used for serving the api. If 0 - equal to the number of available cores.# If missing - Same as storage.max_search_threadsmax_workers: 0# Host to bind the service onhost: 0.0.0.0# HTTP(S) port to bind the service onhttp_port: 6333# gRPC port to bind the service on.# If `null` - gRPC is disabled. Default: null# Comment to disable gRPC:grpc_port: 6334# Enable CORS headers in REST API.# If enabled, browsers would be allowed to query REST endpoints regardless of query origin.# More info: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS# Default: trueenable_cors: true# Enable HTTPS for the REST and gRPC APIenable_tls: false# Check user HTTPS client certificate against CA file specified in tls configverify_https_client_certificate: false# Set an api-key.# If set, all requests must include a header with the api-key.# example header: `api-key: <API-KEY>`## If you enable this you should also enable TLS.# (Either above or via an external service like nginx.)# Sending an api-key over an unencrypted channel is insecure.## Uncomment to enable.# api_key: your_secret_api_key_here# Set an api-key for read-only operations.# If set, all requests must include a header with the api-key.# example header: `api-key: <API-KEY>`## If you enable this you should also enable TLS.# (Either above or via an external service like nginx.)# Sending an api-key over an unencrypted channel is insecure.## Uncomment to enable.# read_only_api_key: your_secret_read_only_api_key_here# Uncomment to enable JWT Role Based Access Control (RBAC).# If enabled, you can generate JWT tokens with fine-grained rules for access control.# Use generated token instead of API key.## jwt_rbac: true# Hardware reporting adds information to the API responses with a# hint on how many resources were used to execute the request.## Uncomment to enable.# hardware_reporting: truecluster:# Use `enabled: true` to run Qdrant in distributed deployment modeenabled: false# Configuration of the inter-cluster communicationp2p:# Port for internal communication between peersport: 6335# Use TLS for communication between peersenable_tls: false# Configuration related to distributed consensus algorithmconsensus:# How frequently peers should ping each other.# Setting this parameter to lower value will allow consensus# to detect disconnected nodes earlier, but too frequent# tick period may create significant network and CPU overhead.# We encourage you NOT to change this parameter unless you know what you are doing.tick_period_ms: 100# Set to true to prevent service from sending usage statistics to the developers.
# Read more: https://qdrant.tech/documentation/guides/telemetry
telemetry_disabled: false# TLS configuration.
# Required if either service.enable_tls or cluster.p2p.enable_tls is true.
tls:# Server certificate chain filecert: ./tls/cert.pem# Server private key filekey: ./tls/key.pem# Certificate authority certificate file.# This certificate will be used to validate the certificates# presented by other nodes during inter-cluster communication.## If verify_https_client_certificate is true, it will verify# HTTPS client certificate## Required if cluster.p2p.enable_tls is true.ca_cert: ./tls/cacert.pem# TTL in seconds to reload certificate from disk, useful for certificate rotations.# Only works for HTTPS endpoints. Does not support gRPC (and intra-cluster communication).# If `null` - TTL is disabled.cert_ttl: 3600
主要修改以下參數:
- ? ?storage_path: E:/qdrant/storage
- ? ?snapshots_path: E:/qdrant/snapshots
- ? ?host: 0.0.0.0
- ? ?http_port: 6333
3.3、生成WebUI資源文件
? 執行以下sh文件,生成static目錄
#!/usr/bin/env bashset -euo pipefailSTATIC_DIR=${STATIC_DIR:-"./static"}
OPENAPI_FILE=${OPENAPI_DIR:-"./docs/redoc/master/openapi.json"}# Download `dist.zip` from the latest release of https://github.com/qdrant/qdrant-web-ui and unzip given folder# Get latest dist.zip, assume jq is installed
DOWNLOAD_LINK=$(curl --silent "https://api.github.com/repos/qdrant/qdrant-web-ui/releases/latest" | jq -r '.assets[] | select(.name=="dist-qdrant.zip") | .browser_download_url')if command -v wget &> /dev/null
thenwget -O dist-qdrant.zip $DOWNLOAD_LINK
elsecurl -L -o dist-qdrant.zip $DOWNLOAD_LINK
firm -rf "${STATIC_DIR}/"*
unzip -o dist-qdrant.zip -d "${STATIC_DIR}"
rm dist-qdrant.zip
cp -r "${STATIC_DIR}/dist/"* "${STATIC_DIR}"
rm -rf "${STATIC_DIR}/dist"cp "${OPENAPI_FILE}" "${STATIC_DIR}/openapi.json"
? 資源文件明細:
注意: 把生成的static文件夾,復制到qdrant.exe文件的同級目錄
四、運行Qdrant Web UI
4.1、啟動Qdrant
?在命令行執行以下命令
qdrant.exe --config-path E:/qdrant/config/config.yaml
4.2、訪問WebUI
http://localhost:6333/dashboard
五、附帶說明
5.1、直接使用qdrant-web-ui
? ? 以上web-ui是集成到qdrant的使用方式,也可以直接使用qdrant-web-ui的方式
https://github.com/qdrant/qdrant-web-uihttps://github.com/qdrant/qdrant-web-ui? 1. 下載qdrant-web-ui源碼
? 2. 安裝依賴
? ? ? npm install
? 3. 啟動開發服務器
? ? ? npm start
? 4. 訪問qdrant-web-ui(端口為5173)
? ? ? 打開瀏覽器訪問 http://localhost:5173/
? 5. 修改默認端口為3000
? ?修改vite.config.js文件
import { defineConfig } from 'vite';
import reactRefresh from '@vitejs/plugin-react';
import svgrPlugin from 'vite-plugin-svgr';
import eslintPlugin from 'vite-plugin-eslint';
import {rehypeMetaAsAttributes} from "./src/lib/rehype-meta-as-attributes";// https://vitejs.dev/config/
export default defineConfig(async () => {const mdx = await import('@mdx-js/rollup');return {base: './',// This changes the output dir from dist to build// comment this out if that isn't relevant for your projectbuild: {outDir: 'dist',},server: {port: 3000 // 修改為你希望使用的端口號},plugins: [reactRefresh(),svgrPlugin({svgrOptions: {icon: true,// ...svgr options (https://react-svgr.com/docs/options/)},}),eslintPlugin({include: ['src/**/*.jsx', 'src/**/*.js', 'src/**/*.ts', 'src/**/*.tsx'],exclude: ['node_modules/**','dist/**, build/**','**/*.mdx','**/*.md'],}),mdx.default({rehypePlugins: [rehypeMetaAsAttributes,],}),],test: {globals: true,environment: 'jsdom',setupFiles: ['./src/setupTests.js'],},}
});