文檔智能：OCR+Rocketqa+layoutxlm ＜Rocketqa＞

此次梳理Rocketqa，個人認為該篇文件講述的是段落搜索的改進點，關于其框架：粗檢索 + 重排序----（dual-encoder architecture），講訴不多，那是另外的文章；

之前根據文檔智能功能，粗略過了一遍。

文檔智能：OCR+Rocketqa+layoutxlm＜LayoutLMv2＞

最近在看RAG相關內容，提到了檢索排序，故而重新梳理。如有不足或錯誤之處，歡迎感謝指正。

記錄如下：

RocketQA是一種優化訓練方法，用于密集段落檢索（Dense Passage Retrieval，DPR），以支持開放域問答（Open-Domain Question Answering，ODQA）系統。

1. Abstract & Introduction

It is difficult to effectively train a dual-encoder for dense passage retrieval due to the following three major challenges:

First, there exists the discrepancy between training and inference for the dual-encoder retriever.
During inference, the retriever needs to identify positive (or relevant) passages for each question from a large collection containing millions of candidates.
However, during training, the model is learned to estimate the probabilities of positive passages in a small candidate set for each question, due to the limited memory of a single GPU (or other device).

To reduce such a discrepancy, previous work tried to design specific mechanisms for selecting a few hard negatives from the top-k retrieved candidates. However, it suffers from the false negative issue due to the following challenge.

Second, there might be a large number of unlabeled positives.

Third, it is expensive to acquire large-scale training data for open-domain QA.

采用的一系列優化策略：跨批次負采樣（Cross-batch Negatives）、去噪的強負例采樣（Denoised Hard Negatives）和數據增強（Data Augmentation）等。

用于解決訓練過程中負例樣本不足，和，存在大量錯誤負例樣本的問題。

First, RocketQA introduces cross-batch negatives. Comparing to in-batch negatives, it increases the number of available negatives for each question during training, and alleviates the discrepancy between training and inference.

Second, RocketQA introduces denoised hard negatives. It aims to remove false negatives from the top-ranked results retrieved by a retriever, and derive more reliable hard negatives.

Third, RocketQA leverages large-scale unsupervised data “labeled” by a cross-encoder (as shown in Figure1b) for data augmentation.

Though inefficient, the cross-encoder architecture has been found to be more capable than the dual-encoder architecture in both theory and practice.

Therefore, we utilize a cross-encoder to generate high quality pseudo labels for unlabeled data which are used to train the dual-encoder retriever.

在這里插入圖片描述

2. Related work

2.1 Passage retrieval for open-domain QA

Recently, researchers have utilized deep learning to improve traditional passage retrievers, including:

document expansions,
question expansions,
term weight estimation.

Different from the above term-based approaches, dense passage retrieval has been proposed to represent both questions and documents as dense vectors (i.e., embeddings), typically in a dual-encoder architecture (as shown in Figure 1a).

在這里插入圖片描述

Existing approaches can be divided into two categories:
(1) self-supervised pre-training for retrieval.
(2) fine-tuning pre-trained language models on labeled data.

Our work follows the second class of approaches, which show better performance with less cost.

2.2 Passage re-ranking for open-domain QA

Based on the retrieved passages from a first-stage retriever, BERT-based rerankers have recently been applied to retrieval-based question answering and search-related tasks, and yield substantial improvements over the traditional methods.
基于從第一階段檢索器檢索到的段落，BERT-based（基于BERT的）重排器最近被應用于基于檢索的問答系統和搜索相關任務，相較于傳統方法，取得了顯著的改進。

Although effective to some extent, these rankers employ the cross-encoder architecture (as shown in Figure 1b) that is impractical to be applied to all passages in a corpus with respect to a question.
盡管在某種程度上是有效的，但這些排序器采用了交叉編碼器架構（如圖1b所示），這對于應用于語料庫中與問題有關的所有段落是不切實際的。

The re-rankers with light weight interaction based on the representations of dense retrievers have been studied. However, these techniques still rely on a separate retriever which provides candidates and representations.
已經研究了基于密集檢索器表示且具有輕量級交互的重排器。然而，這些技術仍然依賴于一個單獨的檢索器來提供候選結果和表示。

As a comparison, we focus on developing dual-encoder based retrievers.

3. Approach

3.1 Task Description

The task of open-domain QA is described as follows.
Given a natural language question, a system is required to answer it based on a large collection of documents.

Let $C$ denote the corpus, consisting of $N$ documents.

We split the $N$ documents into $M$ passages, denoted by $p_{1}$ , $p_{2}$ , …, $p_{M}$ ,

where each passage $p_{i}$ can be viewed as an $l$ -length sequence of tokens $p_{i}^{(1)}$ , $p_{i}^{(2)}$ , …, $p_{i}^{(l)}$ .

Given a question $q$ , the task is to find a passage $p_{i}$ among the $M$ candidates,

and extract a span $p_{i}^{(s)}$ , $p_{i}^{(s+1)}$ , …, $p_{i}^{(e)}$ from $p_{i}$ that can answer the question.

In this paper, we mainly focus on developing a dense retriever to retrieve the passages that contain the answer.

每個段落的長度 $l$ 是同一個數值嗎？

見4.1.3：

4.1.3 Implementation Details

1. Maximal length

We set the maximum length of questions and passages as 32 and 128, respectively.

3.2 The Dual-Encoder Architecture

We develop our passage retriever based on the typical dual-encoder architecture, as illustrated in Figure 1a.

First, a dense passage retriever uses an encoder $E_{p}(·)$ to obtain the $d$ -dimensional real-valued vectors (a.k.a., embedding) of passages.

Then, an index of passage embeddings is built for retrieval.

At query time, another encoder $E_{q}(·)$ is applied to embed the input question to a $d$ -dimensional real-valued vector, and $k$ passages whose embeddings are the closest to the question’s will be retrieved.

The similarity between the question $q$ and a candidate passage $p$ can be computed as the dot product of their vectors:
在這里插入圖片描述

In practice, the separation of question encoding and passage encoding is desirable, so that the dense representations of all passages can be precomputed for efficient retrieval.
在實踐中，將問題編碼和段落編碼分離是理想的做法，因為這樣可以先預先計算出所有段落的密集表示，從而實現高效的檢索。

Here, we adopt two independent neural networks initialized from pre-trained LMs for the two encoders $E_{q}(·)$ and $E_{p}(·)$ separately,
在這里，我們分別為兩個編碼器 Eq(·) 和 Ep(·) 采用了兩個從預訓練語言模型（LMs）初始化的獨立神經網絡，

and take the representations at the first token (e.g., [CLS] symbol in BERT) as the output for encoding.
并取第一個標記（例如，在BERT中的[CLS]符號）的表示作為編碼的輸出。

為什么使用[CLS]符號）的表示作為編碼的輸出，簡單解釋的話，是BERT使用的是transformer結構，而一句話的開始的標記[CLS]能夠“兼顧”整句話的含義。

詳細可以看鏈接：
https://blog.csdn.net/sdsasaAAS/article/details/142926242
https://blog.csdn.net/weixin_45947938/article/details/144232649

3.2.1 Training

Formally, given a question $q_{i}$ together with its positive passage $p_{i}^+$ and $m$ negative passages $\left\{p_{i, j}^-\right\}_{j=1}^m$ , we minimize the loss function:

在這里插入圖片描述

where we aim to optimize the negative log likelihood of the positive passage against a set of $m$ negative passages.

Ideally, we should take all the negative passages in the whole collection into consideration in Equation 2.

However, it is computationally infeasible to consider a large number of negative samples for a question, and hence $m$ is practically set to a small number that is far less than $M$ .

As what will be discussed later, both the number and the quality of negatives affect the final performance of passage retrieval.

3.2.2 Inference

In our implementation, we use FAISS to index the dense representations of all passages.
使用了FAISS（Facebook AI Similarity Search）庫來對所有段落的密集表示進行索引。

Specifically, we use IndexFlatIP for indexing and the exact maximuminner product search for querying.
具體地說，使用了 IndexFlatIP 作為索引類型，以及精確的最大內積搜索（exact maximum inner product search）作為查詢方法。

FAISS：是一個高效相似性搜索和稠密向量聚類的庫，尤其適用于在大型數據集上進行快速相似性搜索。
IndexFlatIP：這是一個基于平坦（flat）索引的FAISS類；
它直接存儲了所有向量，并在查詢時計算查詢向量與所有存儲向量的內積。
IP代表內積（Inner Product），所以 IndexFlatIP 適用于那些需要基于內積相似性度量（如余弦相似度）的應用場景。
最大內積搜索：這是基于內積相似度的一種搜索方法。對于給定的查詢向量，它會找到與查詢向量內積最大的存儲向量。這在信息檢索、推薦系統等領域中特別有用，因為這些領域通常涉及到計算向量之間的相似性。

通過結合使用IndexFlatIP和最大內積搜索，能夠在大型文本集合中高效地找到與給定查詢最相似的段落。

對于更大規模的數據集，可能需要考慮使用FAISS提供的更高效的索引方法，如基于聚類的索引（如IndexIVFPQ）或基于圖的索引（如IndexHNSW），以在保持較高搜索質量的同時提高搜索速度。

不理解，沒用過FAISS

3.3 Optimized Training Approach

Three major challenges in training the dual-encoder based retriever, including:

the training and inference discrepancy,
the existence of unlabeled positives,
limited training data.

3.3.1 Cross-batch Negatives

Assume that there are B questions in a mini-batch on a single GPU, and each question has one positive passage.
在這里插入圖片描述
Figure 2: The comparison of traditional in-batch negatives and our cross-batch negatives when trained on multiple GPUs, where A is the number of GPUs, and B is the number of questions in each min-batch.

With A GPUs (or mini-batches) , we can indeed obtain $A \times B ? 1$ negatives for a given question, which is approximately $A$ times as many as the original number of in-batch negatives.

In this way, we can use more negatives in the training objective of Equation 2, so that the results are expected to be improved.

3.3.2 Denoised Hard Negatives

因為人工標記的標簽是有限的，存在大量未標記的正確答案；所以之前：

To obtain hard negatives, a straightforward method is to select the top-ranked passages (excluding the labeled positive passages) as negative samples.

這種方法，容易假陰；

基于此：

We first train a cross-encoder.

Then, when sampling hard negatives from the top-ranked passages retrieved by a dense retriever, we select only the passages that are predicted as negatives by the cross-encoder with high confidence scores.

The selected top-retrieved passages can be considered as denosied samples that are more reliable to be used as hard negatives.

3.3.3 Data Augmentation

The third strategy aims to alleviate the issue of limited training data.

Since the cross-encoder is more powerful in measuring the similarity between questions and passages, we utilize it to annotate unlabeled questions for data augmentation.

Specifically, we incorporate a new collection of unlabeled questions, while reuse the passage collection.

Then, we use the learned cross-encoder to predict the passage labels for the new questions.

To ensure the quality of the automatically labeled data, we only select the predicted positive and negative passages with high confidence scores estimated by the cross-encoder.

Finally, the automatically labeled data is used as augmented training data to learn the dual encoder.

3.4 The Training Procedure

在這里插入圖片描述
Require:
Let $C$ denote a collection of passages.
$Q_{L}$ is a set of questions that have corresponding labeled passages in $C$ ,
$Q_{U}$ is a set of questions that have no corresponding labeled passages.
$D_{L}$ is a dataset consisting of $C$ and $Q_{L}$ ,
$D_{U}$ is a dataset consisting of $C$ and $Q_{U}$ .

Step1:
Train a dual-encoder $M_{D}^{(0)}$ by using cross-batch negatives on $D_{L}$ .

STEP 2:
Train a cross-encoder $M_{C}$ on $D_{L}$ .

The positives used for training the cross-encoder are from the original training set $D_{L}$ ,
while the negatives are randomly sampled from the top-k passages (excluding the labeled positive passages) retrieved by $M_{D}^{(0)}$ from $C$ for each question $\in D_{L}$ .

This design is to let the cross-encoder adjust to the distribution of the results retrieved by the dual-encoder, since the cross-encoder will be used in the following two steps for optimizing the dual-encoder.

STEP 3:
Train a dual-encoder $M_{D}^{(1)}$ by further introducing denoised hard negative sampling on $D_{L}$ .

Regarding to each question $\in D_{L}$ , the hard negatives are sampled from the top passages retrieved by $M_{D}^{(0)}$ from $C$ ,

and only the passages that are predicted as negatives by the cross-encoder $M_{C}$ with high confidence scores will be selected.

STEP 4:
Construct pseudo training data $D_{U}$ by using $M_{C}$ to label the top-k passages retrieved by $M_{D}^{(1)}$ from $C$ for each question $\in D_{U}$ ,

and then train a dual-encoder $M_{D}^{(2)}$ on both the manually labeled training data $D_{L}$ and the automatically augmented training data $D_{U}$ .

我個人理解為，

先用人工標記的數據集， $D_{L}$ ，訓練一個檢索模型 dual-encoder， $M_{D}^{(0)}$ ；

然后，訓練一個分類模型，cross-encoder， $M_{C}$ ，該模型最后給出正負樣本的二分類。其中，正樣本來自 $D_{L}$ ，負樣本來自： $M_{D}^{(0)}$ 給出的 top-k passages (excluding the labeled positive passages)。

然后，訓練檢索模型 dual-encoder， $M_{D}^{(1)}$ ；其增加的負樣本，仍然來自 $M_{D}^{(0)}$ 給出的 top-k passages (excluding the labeled positive passages)，不過經過了一些篩選，是第二步中經過cross-encoder預測過為負樣本的負樣本；

這樣會排除一些直接使用 $M_{D}^{(0)}$ 給出的 top-k passages (excluding the labeled positive passages)導致的未標記的正樣本；

再然后，將 $D_{U}$ 喂給 $M_{D}^{(1)}$ ，get the top-k passages；將這些數據再喂給 $M_{C}$ 輸出標簽；

然后使用人工標記的 $D_{L}$ ，和，得到“偽標簽”的 $D_{U}$ ，再訓練一個檢索模型 dual-encoder， $M_{D}^{(2)}$ 。

說 $M_{C}$ 是二分類模型是不合適的，結合 4.1.3 來看，其也是個檢索模型：

4.1 Experimental Setup

4.1.3 Implementation Details

1. Pre-trained LMs

The dual-encoder is initialized with the parameters of ERNIE 2.0 base, and the cross-encoder is initialized with ERNIE 2.0 large.

2. Denoised hard negatives and data augmentation

We use the cross-encoder for both denoising hard negatives and data augmentation.

Specifically, we select the top retrieved passages with scores less than 0.1 as negatives and those with scores higher than 0.9 as positives.

We manually evaluated the selected data, and the accuracy was higher than 90%.

3. The number of positives and negatives

When training the cross-encoders, the ratios of the number of positives to the number of negatives are 1:4 and 1:1 on MSMARCO and NQ, respectively.

The negatives used for training cross-encoders are randomly sampled from the top-1000 and top-100 passages retrieved by the dual-encoder $M_{D}^{(0)}$ on MSMARCO and NQ, respectively.

When training the dual-encoders in the last two steps ( $M_{D}^{(1)}$ ? and $M_{D}^{(2)}$ ?), we set the ratios of the number of positives to the number of hard negatives as 1:4 and 1:1 on MSMARCO and NQ, respectively.