---

前言

在之前的文章中，博主大致介紹了3A的作用、使用場景，并且以webrtc開源框架，說明了在webrtc中音頻3A開啟的方法和選項，以及對于3A模塊實例如何塞入近端信號和遠端信號的流程結合代碼做了詳細介紹。

webrtc中的SplittingFilter的主要作用是使用濾波器組進行子帶分割，好為后面做回聲消除做準備，本篇文章將會先介紹SplittingFilter代碼的整體流程，包括創建，分割，和合并的業務代碼。然后再進一步的介紹算法是如何實現以及實現細節。

|版本聲明：山河君，未經博主允許，禁止轉載

一、頻帶分割過程

1.SplittingFilter的創建

從最開始的地方來看一下SplittingFilter的創建時機，如果讀者對于webrtc有一定了解，那么應該知道對于音頻數據是存儲在AudioBuffer中的，AudioBuffer創建時同時會創建SplittingFilter。

AudioBuffer::AudioBuffer(size_t input_rate,size_t input_num_channels,size_t buffer_rate,size_t buffer_num_channels,size_t output_rate,size_t output_num_channels): input_num_frames_(static_cast<int>(input_rate) / 100),......num_bands_(NumBandsFromFramesPerChannel(buffer_num_frames_)), //判斷需要劃分的頻帶{.....if (num_bands_ > 1) { //判斷是否創建SplittingFiltersplit_data_.reset(new ChannelBuffer<float>(buffer_num_frames_, buffer_num_channels_, num_bands_));splitting_filter_.reset(new SplittingFilter(buffer_num_channels_, num_bands_, buffer_num_frames_));}
}

根據上文提到的，對于語音信號，人聲區間應該在0~8kHz，即：

• 對于32k采樣率應該劃分為 0 ~ 8kHz， 8 ~ 16kHz
• 對于48k采樣率劃分到 0 ~ 8kHz， 8 ~ 16kHz，16 ~ 24kHz。

所以對于16k采樣率即不用再劃分頻帶，而NumBandsFromFramesPerChannel(buffer_num_frames_)就是通過采樣率判斷需要劃分的頻帶區間。

2.頻帶分割整體流程

1）分割時機

AudioBuffer會在合適的時機進行頻帶分割，例如近端信號會在以下條件（如開啟高通濾波器，降噪等）滿足時進行分割：

bool AudioProcessingImpl::SubmoduleStates::CaptureMultiBandProcessingActive(bool ec_processing_active) const {return high_pass_filter_enabled_ || mobile_echo_controller_enabled_ ||noise_suppressor_enabled_ || adaptive_gain_controller_enabled_ ||(echo_controller_enabled_ && ec_processing_active);
}if (submodule_states_.CaptureMultiBandSubModulesActive() &&SampleRateSupportsMultiBand(capture_nonlocked_.capture_processing_format.sample_rate_hz())) {capture_buffer->SplitIntoFrequencyBands();}

其中SplitIntoFrequencyBands接口就是開啟進行分割的入口

void AudioBuffer::SplitIntoFrequencyBands() {splitting_filter_->Analysis(data_.get(), split_data_.get());
}

2）分割規則

SplittingFilter根據頻帶選擇是分割成 0 ~ 8kHz， 8 ~ 16kHz還是分割成三份 0 ~ 8kHz， 8 ~ 16kHz，16 ~ 24kHz

void SplittingFilter::Analysis(const ChannelBuffer<float>* data,ChannelBuffer<float>* bands) {.....if (bands->num_bands() == 2) {TwoBandsAnalysis(data, bands); // 0 ~ 8kHz， 8 ~ 16kHz} else if (bands->num_bands() == 3) {ThreeBandsAnalysis(data, bands); // 0 ~ 8kHz， 8 ~ 16kHz，16 ~ 24kHz}
}

3）分割核心代碼

假如分割成兩份，那么核心的代碼就是如下：

void SplittingFilter::TwoBandsAnalysis(const ChannelBuffer<float>* data,ChannelBuffer<float>* bands) {RTC_DCHECK_EQ(two_bands_states_.size(), data->num_channels());RTC_DCHECK_EQ(data->num_frames(), kTwoBandFilterSamplesPerFrame);for (size_t i = 0; i < two_bands_states_.size(); ++i) {std::array<std::array<int16_t, kSamplesPerBand>, 2> bands16;std::array<int16_t, kTwoBandFilterSamplesPerFrame> full_band16;FloatS16ToS16(data->channels(0)[i], full_band16.size(), full_band16.data());WebRtcSpl_AnalysisQMF(full_band16.data(), data->num_frames(),bands16[0].data(), bands16[1].data(),two_bands_states_[i].analysis_state1,two_bands_states_[i].analysis_state2);S16ToFloatS16(bands16[0].data(), bands16[0].size(), bands->channels(0)[i]);S16ToFloatS16(bands16[1].data(), bands16[1].size(), bands->channels(1)[i]);}
}

• 核心代碼過程：
  • FloatS16ToS16對于多通道的情況，先取一個通道的數據，將其從FloatS16轉為int16
  • WebRtcSpl_AnalysisQMF進行頻帶分割，將兩個頻帶音頻數據分別放入bands16[0]和 bands16[1]，two_bands_states_[i].analysis_state存儲不同通道，兩個頻帶的狀態
  • S16ToFloatS16再將兩個頻帶的數據恢復為float類型

3.頻帶合并

有頻帶分割，當然可以把分割的頻帶合并為原始音頻，過程和分割步驟相似

void SplittingFilter::TwoBandsSynthesis(const ChannelBuffer<float>* bands,ChannelBuffer<float>* data) {RTC_DCHECK_LE(data->num_channels(), two_bands_states_.size());RTC_DCHECK_EQ(data->num_frames(), kTwoBandFilterSamplesPerFrame);for (size_t i = 0; i < data->num_channels(); ++i) {std::array<std::array<int16_t, kSamplesPerBand>, 2> bands16;std::array<int16_t, kTwoBandFilterSamplesPerFrame> full_band16;FloatS16ToS16(bands->channels(0)[i], bands16[0].size(), bands16[0].data());FloatS16ToS16(bands->channels(1)[i], bands16[1].size(), bands16[1].data());WebRtcSpl_SynthesisQMF(bands16[0].data(), bands16[1].data(),bands->num_frames_per_band(), full_band16.data(),two_bands_states_[i].synthesis_state1,two_bands_states_[i].synthesis_state2);S16ToFloatS16(full_band16.data(), full_band16.size(), data->channels(0)[i]);}
}

二、算法實現

1.實現原理介紹

webrtc的SplittingFilter源碼中主要應用到以下的原理：

• QMF正交鏡像濾波器：語音信號處理二十九——QMF濾波器設計原理與實現
• 多項分解和noble恒等式：語音信號處理三十——高效多相抽取器
• 全通QMF濾波器：webrtc之子帶分割上——All-pass QMF濾波器

webrtc中是采用三個一階全通濾波器實現的all pass qmf，它的整體結構如下：

輸入: x[n]|↓Even x[2n] → A1(z) -→ A2(z) -→ A3(z) ──┐│      ↓ half_band_low[n]Odd  x[2n+1] → A1(z) -→ A2(z) -→ A3(z) ─┘↓ half_band_high[n][用于語音處理]合成：↑x?[2n]   = low + highx?[2n+1] = low - high

2.All pass QMF系統源碼

void WebRtcSpl_AnalysisQMF(const int16_t* in_data, size_t in_data_length,int16_t* low_band, int16_t* high_band,int32_t* filter_state1, int32_t* filter_state2)
{size_t i;int16_t k;int32_t tmp;int32_t half_in1[kMaxBandFrameLength];int32_t half_in2[kMaxBandFrameLength];int32_t filter1[kMaxBandFrameLength];int32_t filter2[kMaxBandFrameLength];const size_t band_length = in_data_length / 2;RTC_DCHECK_EQ(0, in_data_length % 2);RTC_DCHECK_LE(band_length, kMaxBandFrameLength);// Split even and odd samples. Also shift them to Q10.for (i = 0, k = 0; i < band_length; i++, k += 2){half_in2[i] = ((int32_t)in_data[k]) * (1 << 10);half_in1[i] = ((int32_t)in_data[k + 1]) * (1 << 10);}// All pass filter even and odd samples, independently.WebRtcSpl_AllPassQMF(half_in1, band_length, filter1,WebRtcSpl_kAllPassFilter1, filter_state1);WebRtcSpl_AllPassQMF(half_in2, band_length, filter2,WebRtcSpl_kAllPassFilter2, filter_state2);// Take the sum and difference of filtered version of odd and even// branches to get upper & lower band.for (i = 0; i < band_length; i++){tmp = (filter1[i] + filter2[i] + 1024) >> 11;low_band[i] = WebRtcSpl_SatW32ToW16(tmp);tmp = (filter1[i] - filter2[i] + 1024) >> 11;high_band[i] = WebRtcSpl_SatW32ToW16(tmp);}
}

1）提高精度

根據整體結構，首先是將信號分為奇偶信號，具體代碼如下：

 for (i = 0, k = 0; i < band_length; i++, k += 2){half_in2[i] = ((int32_t)in_data[k]) * (1 << 10);half_in1[i] = ((int32_t)in_data[k + 1]) * (1 << 10);}

值得注意的是* (1 << 10)是為了提高計算精度，webrtc中常用Q10 格式的定點數（左移 10 位），進行高精度的乘法和累加計算，避免精度損失。

2）經過串聯全通濾波器

WebRtcSpl_AllPassQMF(half_in1, band_length, filter1,WebRtcSpl_kAllPassFilter1, filter_state1);WebRtcSpl_AllPassQMF(half_in2, band_length, filter2,WebRtcSpl_kAllPassFilter2, filter_state2);

其中：

• half_in1：偶信號
• half_in2：奇信號
• filter1，filter2：濾波后信號
• WebRtcSpl_kAllPassFilter1：偶信號濾波器系數
• WebRtcSpl_kAllPassFilter2：奇信號濾波器系數
• filter_state1，filter_state2：保存濾波器狀態，實現保持多個數據塊（frame）之間連續

具體內部實現下文中介紹

3）分離后的低頻高頻

根據All pass QMF濾波器設計：
$$\begin{gathered} H_0(z)=\frac{1}{2}[A(z)+A(?z)] \\ {H}_1(z)=\frac{1}{2}[A(z)?A(?z)] \end{gathered}$$
濾波后得到低頻高頻信號：

for (i = 0; i < band_length; i++){tmp = (filter1[i] + filter2[i] + 1024) >> 11;low_band[i] = WebRtcSpl_SatW32ToW16(tmp);tmp = (filter1[i] - filter2[i] + 1024) >> 11;high_band[i] = WebRtcSpl_SatW32ToW16(tmp);}

這里右移11位而不是10位，就是All pass QMF濾波器設計中要除以2

3.濾波算法實現

1）濾波器系數

從上文中，對于低頻子帶和高頻子帶中三個全通濾波器系數分別是：

// QMF filter coefficients in Q16.
static const uint16_t WebRtcSpl_kAllPassFilter1[3] = {6418, 36982, 57261};
static const uint16_t WebRtcSpl_kAllPassFilter2[3] = {21333, 49062, 63010};

項目	內容
WebRtcSpl_kAllPassFilter1	第一組三級一階全通濾波器的 Q16 系數，用于構造低子帶
WebRtcSpl_kAllPassFilter2	第二組三級一階全通濾波器的 Q16 系數，用于構造高子帶
格式	Q16，表示 $\frac{value}{65536}$
來源	來源于經典 QMF 結構設計，用于語音信號的二分頻濾波處理（Subband splitting）

通過matlab畫出來，其$A(z)$和$A(?z)$的相頻響應是如下圖中所示：

2）全通濾波器串聯

void WebRtcSpl_AllPassQMF(int32_t* in_data, size_t data_length,int32_t* out_data, const uint16_t* filter_coefficients,int32_t* filter_state)
{// The procedure is to filter the input with three first order all pass filters// (cascade operations).////         a_3 + q^-1    a_2 + q^-1    a_1 + q^-1// y[n] =  -----------   -----------   -----------   x[n]//         1 + a_3q^-1   1 + a_2q^-1   1 + a_1q^-1//// The input vector |filter_coefficients| includes these three filter coefficients.// The filter state contains the in_data state, in_data[-1], followed by// the out_data state, out_data[-1]. This is repeated for each cascade.// The first cascade filter will filter the |in_data| and store the output in// |out_data|. The second will the take the |out_data| as input and make an// intermediate storage in |in_data|, to save memory. The third, and final, cascade// filter operation takes the |in_data| (which is the output from the previous cascade// filter) and store the output in |out_data|.// Note that the input vector values are changed during the process.size_t k;int32_t diff;// First all-pass cascade; filter from in_data to out_data.// Let y_i[n] indicate the output of cascade filter i (with filter coefficient a_i) at// vector position n. Then the final output will be y[n] = y_3[n]// First loop, use the states stored in memory.// "diff" should be safe from wrap around since max values are 2^25// diff = (x[0] - y_1[-1])diff = WebRtcSpl_SubSatW32(in_data[0], filter_state[1]);// y_1[0] =  x[-1] + a_1 * (x[0] - y_1[-1])out_data[0] = WEBRTC_SPL_SCALEDIFF32(filter_coefficients[0], diff, filter_state[0]);// For the remaining loops, use previous values.for (k = 1; k < data_length; k++){// diff = (x[n] - y_1[n-1])diff = WebRtcSpl_SubSatW32(in_data[k], out_data[k - 1]);// y_1[n] =  x[n-1] + a_1 * (x[n] - y_1[n-1])out_data[k] = WEBRTC_SPL_SCALEDIFF32(filter_coefficients[0], diff, in_data[k - 1]);}// Update states.filter_state[0] = in_data[data_length - 1]; // x[N-1], becomes x[-1] next timefilter_state[1] = out_data[data_length - 1]; // y_1[N-1], becomes y_1[-1] next time// Second all-pass cascade; filter from out_data to in_data.// diff = (y_1[0] - y_2[-1])diff = WebRtcSpl_SubSatW32(out_data[0], filter_state[3]);// y_2[0] =  y_1[-1] + a_2 * (y_1[0] - y_2[-1])in_data[0] = WEBRTC_SPL_SCALEDIFF32(filter_coefficients[1], diff, filter_state[2]);for (k = 1; k < data_length; k++){// diff = (y_1[n] - y_2[n-1])diff = WebRtcSpl_SubSatW32(out_data[k], in_data[k - 1]);// y_2[0] =  y_1[-1] + a_2 * (y_1[0] - y_2[-1])in_data[k] = WEBRTC_SPL_SCALEDIFF32(filter_coefficients[1], diff, out_data[k-1]);}filter_state[2] = out_data[data_length - 1]; // y_1[N-1], becomes y_1[-1] next timefilter_state[3] = in_data[data_length - 1]; // y_2[N-1], becomes y_2[-1] next time// Third all-pass cascade; filter from in_data to out_data.// diff = (y_2[0] - y[-1])diff = WebRtcSpl_SubSatW32(in_data[0], filter_state[5]);// y[0] =  y_2[-1] + a_3 * (y_2[0] - y[-1])out_data[0] = WEBRTC_SPL_SCALEDIFF32(filter_coefficients[2], diff, filter_state[4]);for (k = 1; k < data_length; k++){// diff = (y_2[n] - y[n-1])diff = WebRtcSpl_SubSatW32(in_data[k], out_data[k - 1]);// y[n] =  y_2[n-1] + a_3 * (y_2[n] - y[n-1])out_data[k] = WEBRTC_SPL_SCALEDIFF32(filter_coefficients[2], diff, in_data[k-1]);}filter_state[4] = in_data[data_length - 1]; // y_2[N-1], becomes y_2[-1] next timefilter_state[5] = out_data[data_length - 1]; // y[N-1], becomes y[-1] next time
}

其實算法實現已經在注釋里了，這里對于實現進行介紹，總共有三級，都是一階全通濾波器：

    //         a_3 + q^-1    a_2 + q^-1    a_1 + q^-1// y[n] =  -----------   -----------   -----------   x[n]//         1 + a_3q^-1   1 + a_2q^-1   1 + a_1q^-1//

根據一階全通濾波器的傳遞函數：
$$\begin{gathered} H(q)=\frac{a+q^{?1}}{1+a{q}^{?1}} \\ ?(1+a{q}^{?1})y[n]=(a+q^{?1})x[n] \\ ?y[n]+ay[n?1]=ax[n]+x[n?1] \\ ?y[n]=x[n?1]+a?(x[n]?y[n?1]) \end{gathered}$$
所以對應代碼實現：

• 每一個全通濾波器的初始狀態使用的是上一次的frame對應全通濾波器的輸入輸出，例如：
  • filter_state[0]: $x[?1]$
  • filter_state[1]: $y[?1]$
• 計算$x[n]?y[n?1]$:diff = WebRtcSpl_SubSatW32(in_data[0], filter_state[1]);
• 計算$y[n]$:out_data[0] = WEBRTC_SPL_SCALEDIFF32(filter_coefficients[0], diff, filter_state[0]);

---

總結

本篇博客中介紹了SplittingFilter的作用和使用它的意義，通過對于webrtc源代碼的解讀對于頻帶分割的流程做了深入介紹，需要結合上一篇對于All pass QMF原理的文章一起看才能理解為何要如此設計。

如果對您有所幫助，請幫忙點個贊吧！