是基于LangChain框架下的開發，所以最開始請先

pip install Langchain
pip install bilibili-api-python

技術要點：

1. 使用Langchain框架自帶的Document loaders

2. 修改BiliBiliLoader的源碼，自帶的并不支持當前b站的視頻加載

源碼文件修改：

import json
import re
import warnings
from typing import List, Tupleimport requests
from langchain_core.documents import Document
from bilibili_api import sync, video
from langchain_community.document_loaders.base import BaseLoader
# Pre-compile regular expressions for video ID extraction
BV_PATTERN = re.compile(r"BV\w+")
AV_PATTERN = re.compile(r"av[0-9]+")class BiliBiliLoader(BaseLoader):"""Loader for fetching transcripts from BiliBili videos."""def __init__(self, video_urls: List[str], sessdata: str, bili_jct: str, buvid3: str):"""Initialize with bilibili url.Args:video_urls (List[str]): List of BiliBili video URLs.sessdata (str): SESSDATA cookie value for authentication.bili_jct (str): BILI_JCT cookie value for authentication.buvid3 (str): BUVI3 cookie value for authentication."""self.video_urls = video_urlsself.credential = video.Credential(sessdata=sessdata, bili_jct=bili_jct, buvid3=buvid3)def load(self) -> List[Document]:"""Load and return a list of documents containing video transcripts.Returns:List[Document]: List of Document objects transcripts and metadata."""results = []for url in self.video_urls:transcript, video_info = self._get_bilibili_subs_and_info(url)doc = Document(page_content=transcript, metadata=video_info)results.append(doc)return resultsdef _get_bilibili_subs_and_info(self, url: str) -> Tuple[str, dict]:"""Retrieve video information and transcript for a given BiliBili URL.Args:url (str): BiliBili video URL.Returns:Tuple[str, dict]: A tuple containing the transcript and video information."""bvid = BV_PATTERN.search(url)if bvid:v = video.Video(bvid=bvid.group(), credential=self.credential)else:aid = AV_PATTERN.search(url)if aid:v = video.Video(aid=int(aid.group()[2:]), credential=self.credential)else:raise ValueError(f"Unable to find a valid video ID in URL: {url}")video_info = sync(v.get_info())video_info.update({"url": url})sub = sync(v.get_subtitle(video_info["cid"]))# Retrieve and process subtitle contentsub_list = sub["subtitles"]if sub_list:sub_url = sub_list[0]["subtitle_url"]if not sub_url.startswith("http"):sub_url = "https:" + sub_urlresponse = requests.get(sub_url)if response.status_code == 200:raw_sub_titles = json.loads(response.content)["body"]raw_transcript = " ".join([c["content"] for c in raw_sub_titles])raw_transcript_with_meta_info = (f"Video Title: {video_info['title']}, "f"description: {video_info['desc']}\n\n"f"Transcript: {raw_transcript}")return raw_transcript_with_meta_info, video_infoelse:warnings.warn(f"Failed to fetch subtitles for {url}. "f"HTTP Status Code: {response.status_code}")return "", video_infoelse:warnings.warn(f"No subtitles found for video: {url}. Returning empty transcript.")return "", video_info

其中SESSDATA，BUVID3，BILI_JCT 三個參數需要通過訪問登錄B站進行獲取。固定值也是必須值，不需要刷新，永久有效，但是如果該賬戶訪問次數過多和頻繁存在被封禁情況，現在不知道b站那邊封禁策略。

獲取方法：打開b站網頁，F12開發者工具，應用程序->cookies>www.bilibili.com 下的元素獲取。

from langchain_community.document_loaders import BiliBiliLoader
SESSDATA = "***************************************"
BUVID3 = "**************************************"
BILI_JCT = "******************************************"loader = BiliBiliLoader(["https://www.bilibili.com/video/BV1PZ421S7VF/?spm_id_from=333.1007.tianma.1-2-2.click"
],
sessdata = SESSDATA,
bili_jct = BILI_JCT,
buvid3 = BUVID3,
)
docs = loader.load()
print(docs)

源碼解析：

_get_bilibili_subs_and_info：

他是一個檢索給定 BiliBili URL 的視頻信息和文字記錄。

獲取到視頻信息后，可以找到字幕URL的獲取路徑，訪問該路徑可以獲取到字幕信息：

sub_list = sub["subtitles"]
if sub_list:sub_url = sub_list[0]["subtitle_url"]if not sub_url.startswith("http"):sub_url = "https:" + sub_urlresponse = requests.get(sub_url)

例子：https://aisubtitle.hdslb.com/bfs/ai_subtitle/prod/125040837614317115816310f6f57f99190f192792b6f2d98ac0?auth_key=1708498531-6e1797becb564b90a29714989167da05-0-e9073436bc93efbbb4f87a3b0c3f7b3f

如果請求字幕接口成功，那么對于所有的返回的語音字幕文字進行處理：

response = requests.get(sub_url)
if response.status_code == 200:raw_sub_titles = json.loads(response.content)["body"]raw_transcript = " ".join([c["content"] for c in raw_sub_titles])raw_transcript_with_meta_info = (f"Video Title: {video_info['title']}, "f"description: {video_info['desc']}\n\n"f"Transcript: {raw_transcript}")

問題：

1. 部分b站視頻不支持語音文字字幕獲取，獲取時給錯誤提示，現在測試情況95%的是視頻都是可以獲取到了。

2. 目前視頻語言字幕抓取語言，測試只抓取中文，抓取的數據目測是b站提供的字幕數據，其他國家的語言不提供一律轉為中文。

3. AI 字幕需要使用登錄賬號的cookie進行請求，請求頻繁或者過多會出現封禁，體現為接口返回正常返回內容，但是subtitle列表為空（即使實際上有ai字幕），無法獲取subtitle_url。建議準備多個賬號備用。