Michael Johns, Zhenyu Wang, Bruno Dupont, and Luca Fiaschi

邁克爾·約翰斯，王振宇，布魯諾·杜邦和盧卡·菲亞斯基

“If you can’t measure it, you can’t manage it, or fix it”

“如果無法衡量，就無法管理或修復它”

–Mike Bloomberg

–麥克·彭博(Mike Bloomberg)

Knowing where to allocate marketing dollars and how much to spend is a perennial business problem. The complexity of modern marketing only adds to this challenge. Contemporary measurement methods rely heavily on data from online web-tracking (cookies) that provide only a limited view of advertising touchpoints in the customer journey and could be further jeopardized by new privacy regulations (Juneau, 2020). To get a comprehensive picture of how well marketing budgets are working requires an approach that can account for online (e.g. search, social media, etc.) and offline (TV, radio, etc.) marketing activities with both direct and indirect effects. Media Mix Modeling (MMM) provides one solution to this problem.

知道在哪里分配營銷美元以及花費多少是一個長期的業務問題。 現代營銷的復雜性只會增加這一挑戰。 當代的測量方法嚴重依賴于來自在線網絡跟蹤(cookie)的數據，這些數據僅提供了有限的客戶旅程中廣告接觸點的視圖，并且可能會受到新的隱私法規的進一步危害(Juneau，2020) 。 要全面了解營銷預算的運作情況，需要一種方法，該方法可以說明具有直接和間接影響的在線(例如搜索，社交媒體等)和離線(電視，廣播等)營銷活動。 媒體混合建模(MMM)提供了解決此問題的一種方法。

This post describes how we built a Media Mix Model of customer acquisition to optimize a yearly budget in the hundreds of millions of dollars. We describe the model, some of the challenges we faced when building it, and discuss how it is used to guide marketing strategy.

這篇文章介紹了我們如何建立客戶獲取的媒體混合模型，以優化每年數億美元的預算。 我們將描述該模型，以及在構建模型時我們面臨的一些挑戰，并討論如何將其用于指導營銷策略。

什么是營銷組合模型(MMM)？ (What is a Marketing Mix Model (MMM)?)

Media mix modeling is a statistical modeling technique for quantifying the effectiveness of advertising on business metrics like new customer acquisitions. MMMs have been in use since the 1960’s (e.g., Borden, 1964) and are common in many industries.

媒體組合建模是一種統計建模技術，用于量化廣告在諸如新客戶獲取等業務指標上的有效性。 MMM自1960年代開始使用(例如Borden，1964)，并且在許多行業中很普遍。

Our MMM is designed to estimate the incremental impact of a marketing channel (think Facebook, podcasting, online display ads) on the number of new subscribers. These estimates can be used to better understand and optimize the efficiency of different allocations of our marketing budget (media mix). The MMM is especially helpful in quantifying the impact of offline channels like television, billboards, or radio advertising, which are difficult to assess using digital measurement solutions.

我們的MMM旨在估算營銷渠道(例如Facebook，播客，在線展示廣告)對新訂戶數量的增量影響。 這些估算值可用于更好地理解和優化我們營銷預算(媒體組合)不同分配的效率。 MMM在量化脫機頻道(如電視，廣告牌或廣播廣告)的影響方面特別有用，這些脫機頻道難以使用數字測量解決方案進行評估。

我們如何建立MMM (How we built our MMM)

We developed our model based on the approach described in Jin et al. (2017). They propose using Bayesian methods to build a multivariable regression model with transformations on marketing activity variables (e.g., spending) which account for diminishing returns and lagged effects of impressions. Using Bayesian methods gives us the ability to incorporate our prior knowledge about marketing effects into the model and produce results that are easier to use in practice while being consistent with field experiments such as lift tests.

我們基于Jin等人中描述的方法開發了模型。 (2017) 。 他們建議使用貝葉斯方法建立一個多變量回歸模型，對營銷活動變量(例如支出)進行轉換，從而解釋收益遞減和印象滯后效應。 使用貝葉斯方法使我們能夠將我們對營銷效果的先驗知識整合到模型中，并產生易于在實踐中使用的結果，同時與現場測試(例如提升測試)保持一致。

In the core modeling framework, marketing dollars spent each week in specific channels (e.g., Hulu, Facebook, direct mail, banner ads, etc.) are used to predict the total number of new customers acquired that week. Spend per channel is transformed using a saturation function to capture diminishing returns on advertising. Channels that can have decaying effects over time, like TV, are further transformed with a function to capture such lagged effects. Control variables are included to account for external factors that can also influence the number of customers signing up each week, such as seasonal variation and discount offers in circulation.

在核心建模框架中，每周在特定渠道(例如Hulu，Facebook，直接郵件，橫幅廣告等)上花費的營銷費用用于預測該周獲得的新客戶總數。 使用飽和度函數轉換每個渠道的支出，以捕獲廣告收益遞減的情況。 隨時間推移可能具有衰減效果的頻道(例如電視)會進一步轉換為具有捕獲此類滯后效果的功能。 包含控制變量以說明可能也會影響每周簽約客戶數量的外部因素，例如季節性變化和流通中的折扣優惠。

In statistical terms, the model has the following form:

用統計術語來說，該模型具有以下形式：

where x_mt is the spend value for a marketing channel m in week t. Spend is transformed using a saturation and decay function f( ); β_m is the effect of channel m on customer acquisition; z_ct is the value of control variable c in week t; β_c is the effect of control variable c; e_t is a Normal error term with mean 0 and constant variance.

其中x_mt是第t周內營銷渠道m的支出值。 使用飽和度和衰減函數f()轉換支出； β_m是渠道m對客戶獲取的影響； z_ct是第t周中控制變量c的值； β_c是控制變量c的作用； e_t是具有均值0和恒定方差的正態誤差項。

This means that the number of new customers acquired each week (y_t) is modeled using weekly marketing spend for the different channels while controlling for factors external to marketing. The model produces a coefficient (β_m) for each channel that represents the number of customers that are acquired for each dollar spent, holding all other spending constant. These coefficients are transformed into the total number of new customers that a channel produces. We can then divide the amount spent on the channel by the estimate of customers acquired to get the incremental customer acquisition cost (iCAC), a standard measure of marketing efficiency.

這意味著，每周的新客戶數量( y_t )是使用不同渠道的每周營銷支出建模的，同時控制了營銷外部的因素。 該模型為每個渠道產生一個系數( β_m )，該系數代表每花費1美元獲得的客戶數量，而使所有其他花費保持不變。 這些系數轉化為渠道產生的新客戶總數。 然后，我們可以將花費在渠道上的金額除以獲得的客戶的估算值，以獲得增量的客戶獲得成本 ( iCAC )，這是營銷效率的標準衡量標準。

Traditional statistical models would assume that the relationship between marketing and customer acquisition follows a straight line. This assumption is often too simplistic. The yield from marketing dollars will tend to approach a point of saturation as spending increases. That is, marketing dollars will start to show diminishing returns after reaching a certain spend level. To account for this saturation, marketing spend is transformed using a nonlinear function that can capture the diminishing returns on advertising:

傳統的統計模型會假設營銷和客戶獲取之間的關系是一條直線。 這種假設通常過于簡單。 隨著支出的增加，營銷美元的收益將趨于接近飽和點。 也就是說，在達到一定的支出水平之后，營銷美元將開始顯示收益遞減。 為了解決這種飽和問題，使用了可以捕獲廣告收益遞減的非線性函數來轉換營銷支出：

where μ is the half-saturation point and x?? is the spend value. The shape of this curve, which is determined by the parameter μ, is learned for each channel as part of the model fitting process. Figure 1 illustrates the shape of the logistic function for values of μ, ranging from 1 to 7.

其中， μ是半飽和點， x??是花費值。 在模型擬合過程中，將為每個通道學習由參數μ確定的曲線形狀。 圖1說明了μ值從1到7的對數函數的形狀。

The ability to learn this curve for each input channel while fitting the model is yet another advantage of Bayesian methods.

貝葉斯方法的另一個優點是能夠在擬合模型的同時為每個輸入通道學習此曲線。

From a marketing perspective, the channel saturation curve allows us to identify the range of spend where we will get the most “bang” for our marketing dollars. The curve can also be plotted against the estimated CAC,(x?? / β? f(x??)), to help understand how quickly efficiency will decrease as spending increases. This pattern can be seen in Figure 2: the CAC generally increases — an indicator of lower efficiency — as spending increases.

從營銷的角度來看，渠道飽和度曲線使我們能夠確定支出范圍，在該范圍內，我們可以最大程度地獲得營銷收入。 該曲線也可以針對估計CAC繪制，(X??/β?F(x??))，以幫助理解效率會如何Swift降低的支出增加。 在圖2中可以看到這種模式：隨著支出的增加，CAC通常會增加，這是效率降低的指標。

In addition to saturation effects, the impact of offline marketing is often distributed over time. Marketing channels with these sorts of decaying effects are further transformed using an adstock function. Advertising adstock is the idea that the impact of advertising peaks at a certain point and continues to have a successively weaker effect for some time period after that peak. We use a geometric decay weight that assumes the effect of advertising peaks at the time of exposure and then decreases over time. The weight function controls the rate of decay through the ɑlpha parameter.

除飽和效應外，離線營銷的影響通常會隨著時間而分布。 使用adstock函數可以進一步轉換具有這種衰減效果的營銷渠道。 廣告廣告資源的想法是，廣告的影響在某個點達到頂峰，并在該頂峰后的某個時間段內持續產生較弱的影響。 我們使用幾何衰減權重，該權重假設在曝光時出現廣告峰值，然后隨著時間的推移而降低。 權重函數通過ɑlpha參數控制衰減率。

Figure 3 below illustrates the shape of the adstock function with different assumptions about how slowly the impact of media exposure can decay.

下面的圖3展示了adstock函數的形狀，并假設了媒體曝光的影響衰減的緩慢程度。

A longer adstock means that the effect of advertising persists longer. The length of the adstock is also learned during the model fitting process. Adstock decay is typical for channels like television or radio — the type of advertising that produces multiple impressions over time. The length of the adstock decay gives us additional insight into how a channel functions. A short decay can suggest a short product consideration time; a long decay suggests that it takes some time before prospective customers decide to adopt the product.

較長的廣告資源意味著廣告效果會持續更長的時間。 還可以在模型擬合過程中了解廣告素材的長度。 廣告庫存衰減是電視或廣播等渠道的典型現象，這種廣告類型會隨著時間的推移產生多次展示。 廣告素材衰減的時間長度使我們對渠道如何發揮作用有了更多的了解。 短暫的衰變可能意味著較短的產品考慮時間。 長時間的衰退表明潛在客戶決定采用該產品需要一些時間。

營銷渠道會計 (Accounting for the Marketing Funnel)

The mixture of marketing channels is often organized into a funnel. Figure 4 illustrates how we might organize offline and online marketing channels based on our knowledge of the way different types of advertising can influence the customer journey.

營銷渠道的混合通常組織成一個渠道。 圖4說明了我們如何根據對不同類型廣告可能影響客戶旅程的方式的了解來組織離線和在線營銷渠道。

Offline, brand channels are typically placed at the top of the funnel; online, direct response channels tend to fall at the bottom. The idea is that channels at the top of the funnel create awareness and interest that drives prospective customers to online channels at the bottom of the funnel before converting.

離線品牌頻道通常位于渠道頂部； 在線直接響應渠道往往落在底部。 想法是，渠道頂部的渠道可以提高知名度和興趣度，從而在轉化之前將潛在客戶吸引到渠道底部的在線渠道。

One implication of the funnel structure is that offline marketing can have a significant downstream effect on certain online channels, a phenomenon referred to as a “funnel effect”. For example, after hearing a podcast ad, a prospective customer would need to go online, where they might encounter search engine marketing. Including channels in the model from all levels of the funnel implies that the amount spent in each channel can be equally impactful in driving conversions. In reality, the amount spent on channels like Search Engine Marketing is more often a product of spending on brand marketing higher up in the funnel (e.g. on TV). Failing to consider these hierarchical relationships in the model could produce misleading results.

渠道結構的一個含義是，離線營銷可以對某些在線渠道產生重大的下游影響，這種現象被稱為“渠道效應”。 例如，在聽到播客廣告后，潛在客戶可能需要上線，在那里他們可能會遇到搜索引擎營銷。 在渠道的各個級別中將模型中的渠道包括在內，意味著在每個渠道中花費的金額對推動轉化的影響同等。 實際上，在搜索引擎營銷等渠道上花費的金額通常是渠道(例如電視)上的品牌營銷支出的產物。 不考慮模型中的這些層次關系可能會產生誤導性的結果。

To deal with these potential funnel effects we estimate the efficiency of marketing spend in two steps. In the first step we fit the core MMM (described above and depicted in Figure 5) while excluding spend from a set of mediating channels at the bottom of the funnel: search engine marketing, affiliate marketing, and online leads. Leaving these downstream channels out of the model gives channels further up in the funnel the opportunity to get credit for the downstream impacts that would otherwise be obscured. We call this the direct model.

為了處理這些潛在的渠道效應，我們分兩步估算營銷支出的效率。 第一步，我們調整核心MMM(如上所述并在圖5中進行了描述)，同時從渠道底部的一組中介渠道( 搜索引擎營銷 ， 會員營銷和在線 銷售 線索 )中排除了支出。 將這些下游渠道排除在模型之外，可以使渠道中更遠的渠道有機會獲得本來可以避免的下游影響的信譽。 我們稱其為直接模型 。

In the second step, we then use the same set of marketing channels to model the amount of weekly spend (not conversions) for search engine marketing, affiliate marketing, and online leads. The coefficients from this lower-funnel model tell us how much marketing spend in the “top” of the funnel is absorbed by these channels at the “bottom” of the funnel. We call this the undirect model and is depicted in Figure 6.

在第二步中，我們然后使用同一組營銷渠道來模擬每周支出金額(而非轉化) 用于搜索引擎營銷，會員營銷和在線潛在客戶 。 這個較低渠道模型的系數告訴我們，渠道“頂部”的渠道吸收了渠道“頂部”的營銷支出。 我們稱其為非直接模型 ， 如圖6所示。

Spend estimates from the directed model are then corrected using the undirected one, producing a funnel-adjusted CAC. Calculating the CAC using the adjusted spend provides a more accurate picture of each channel’s marketing efficiency. Figure 6 shows how the CACs can shift following adjustment for lower-funnel impacts.

然后，使用無向模型校正有向模型的支出估算值，從而產生漏斗調整后的CAC。 使用調整后的支出計算CAC，可以更準確地了解每個渠道的營銷效率。 圖6顯示了CAC如何針對調整后的漏斗影響而轉移。

The mathematical formulation of our fully adjusted CAC* for each time-step t and upper-funnel channel m is:

對于每個時間步長t和上漏斗通道m ，我們完全調整的CAC *的數學公式為：

Where η_mt is a spend adjustment obtained by calculating the effect of the upper funnel channel on the lower funnel spend. And ξ_mt is a seasonal adjustment to conversions obtained by redistributing the extra conversions from seasonal variables. These coefficients are derived from the models for total conversions (y_t, direct model) and total lower funnel spend (undirect model) as follows:

其中η_mt 是通過計算上層渠道渠道對下層渠道支出的影響而獲得的支出調整。 和ξ_mt 是對轉化的季節性調整，該轉化是通過重新分配季節性變量的額外轉化獲得的。 這些系數是從總轉化模型( y_t ，直接模型)和總漏斗支出總額( 非直接模型) 得出的 ，如下所示：

使模型可解釋且可行 (Making the Model Interpretable and Actionable)

A major goal that guided our approach was ensuring that the results would be plausible, interpretable, and actionable. Outputs from inferential models — models that describe relationships — can be difficult to understand and apply in practice. Using a Bayesian modeling approach provides several advantages in accomplishing this goal.

指導我們方法的一個主要目標是確保結果是合理的，可解釋的和可操作的 。 推論模型(描述關系的模型)的輸出可能難以理解并難以在實踐中應用。 使用貝葉斯建模方法可在實現此目標方面提供多個優勢。

Fitting a Bayesian model requires first specifying a prior on the model coefficient being estimated. The prior defines a range of possible values for the channel estimate and a proposal for how likely those values are to occur. It is essentially an educated guess about how much a particular channel influences conversions. The model then uses the observed marketing data to update the proposal encoded in the prior. The most likely value after updating the priors is our best estimate for the marketing channel coefficient.

擬合貝葉斯模型需要首先指定要估計的模型系數的先驗 。 先驗定義了用于信道估計的可能值的范圍以及關于這些值出現的可能性的建議。 從本質上來說，這是對某個特定渠道會影響轉化的有根據的猜測。 然后，該模型使用觀察到的營銷數據來更新先前編碼的提案。 更新先驗后最有可能的價值是我們對營銷渠道系數的最佳估計。

Specifying a prior for each channel estimate makes it possible to set constraints on the results to ensure they fall within a sensible range. For example, it is highly unlikely that marketing would ever have a negative impact on customer acquisition. We, therefore, constrain all channel estimates to be non-negative.

為每個通道估計指定一個先驗，可以對結果設置約束以確保它們落在合理范圍內。 例如，營銷不太可能對客戶獲取產生負面影響。 因此，我們將所有通道估計約束為非負值。

With the prior, we can also use information from lift tests or incrementality analyses to tune model estimates and make them consistent with external benchmarks. Once built, the model is then continually evaluated and adjusted as we conduct tests and get new information about the effectiveness of our marketing. This makes the MMM extremely adaptable to changes in the marketing landscape. We can also incorporate feedback from channel managers to adjust estimates that are unrealistically large or small. As a result, we can provide outputs that are consistent with existing knowledge and responsive to stakeholder needs.

借助先前版本，我們還可以使用提升測試或增量分析中的信息來調整模型估算值，并使它們與外部基準保持一致。 建立模型后，我們將在進行測試并獲得有關營銷有效性的新信息時，對模型進行持續評估和調整。 這使得MMM非常適應營銷環境的變化。 我們還可以結合渠道經理的反饋來調整不切實際的大小估算值。 結果，我們可以提供與現有知識一致并響應利益相關者需求的輸出。

在PyMC3中構建貝葉斯MMM (Building a Bayesian MMM in PyMC3)

The sample code below illustrates how to implement a simple MMM with priors and transformation functions using PyMC3. For this toy example, we assume that there are three marketing channels (X1, X2, X3) and one control variable (Z1). Each marketing channel is transformed using a saturation function to model diminishing returns.

以下示例代碼說明了如何使用PyMC3通過先驗和轉換功能實現簡單的MMM。 對于此玩具示例，我們假設存在三個營銷渠道(X1，X2，X3)和一個控制變量( Z1 )。 使用飽和度函數對每個營銷渠道進行轉換，以建立收益遞減模型。

Building a model requires first defining priors on the unknown parameters to be estimated: intercept; marketing channel, beta; control variable, c_beta; saturation curve shape parameter, half_sat; likelihood noise, sigma. Figure 7 shows the shape of key prior distributions, which are based on the recommendations of Jin et al. (2017).

構建模型要求首先的未知參數先驗定義要估計： 截距 ; 營銷渠道， 測試版 ； 控制變量c_beta ; 飽和曲線形狀參數half_sat ; 似然噪聲， sigma 。 圖7顯示了關鍵先驗分布的形狀，這些分布基于Jin等人的建議。 (2017)。

We next define the model of the expected number of activations, mu, as a linear combination of baseline conversions, the spend in the marketing channels, and the impact of the control variable. This model defines the mean of the likelihood function for the outcome, y_obs, which is represented as a normal distribution with mean μ and variance σ in a linear regression model. Once the model is defined, we sample from the resulting probability space to produce the posterior distributions of the model parameters. Figure 8 shows a sample trace plot that displays the history of the sampling process for a model parameter.

接下來，我們將預期激活次數mu的模型定義為基線轉化，營銷渠道支出和控制變量影響的線性組合。 該模型定義了結果的似然函數y_obs的平均值，在線性回歸模型中以平均值μ和方差σ表示為正態分布。 定義模型后，我們將從結果概率空間中采樣以產生模型參數的后驗分布。 圖8顯示了一個示例軌跡圖，該軌跡圖顯示了模型參數的采樣過程的歷史記錄。

The inspection of the trace plot is used to confirm that the sampling process was able to effectively explore the probability space and converge on a valid distribution. Additionally, it is typical to check model performance by conducting a posterior predictive check (PPC). Data are simulated from the fitted model and compared to the observed data. Variation in the simulated data can be used to build plausibility intervals that are similar in spirit to a confidence interval. Figure 9 shows a sample PPC with a 95% plausibility interval for predicted and observed data.

跟蹤圖的檢查用于確認采樣過程能夠有效地探索概率空間并收斂于有效分布。 此外，通常通過進行后驗預測檢查 (PPC)來檢查模型性能。 從擬合模型模擬數據，并將其與觀察到的數據進行比較。 模擬數據的變化可用于建立在本質上與置信區間相似的合理區間。 圖9顯示了具有95％可信區間的樣本PPC，用于預測和觀察的數據。

Inferences for the target variable and the parameters can then be extracted using the mean (or median) and area of highest probability density of the posterior distributions.

然后可以使用后驗分布的均值(或中位數)和最高概率密度區域來提取目標變量和參數的推論。

使用MMM (Putting the MMM to Use)

MMMs have several applications. As noted, a common use is quantifying the effectiveness and efficiency of marketing. This is particularly important for monitoring offline channels that are difficult to measure on a continuous basis. It also helps us identify lower funnel online channels that tend to absorb some of the influence of upper funnel channels. Taken as a whole, the model gives us an integrated view of channel performance and inter-channel dynamics.

MMM具有多個應用程序。 如前所述，通常的用途是量化營銷的有效性和效率。 這對于監視難以連續測量的脫機頻道特別重要。 它還有助于我們確定下層渠道在線渠道，這些渠道傾向于吸收上層渠道的某些影響。 總體而言，該模型為我們提供了渠道性能和渠道間動態的綜合視圖。

While an MMM can help fill knowledge gaps, the overarching purpose of the model is to provide insights that can inform strategic planning. We use iCAC estimates from the model to identify channels that are underperforming, as well as those that are highly efficient. Spend can then be shifted between channels to produce an overall mix that maximizes the total efficiency of our marketing.

盡管MMM可以幫助填補知識空白，但是該模型的總體目的是提供可以為戰略規劃提供依據的見解。 我們使用模型中的iCAC估算值來確定效果不佳的渠道以及高效渠道。 然后可以在渠道之間轉移支出，以產生整體組合，從而最大限度地提高營銷的整體效率。

To do this, we plug model coefficients into a constrained optimization algorithm to generate recommendations for how to distribute our marketing budget, B, to maximize the number of conversions in a certain period of time:

為此，我們將模型系數插入受約束的優化算法中，以生成有關如何分配營銷預算B的建議 ，以在特定時間段內最大化轉化次數：

The optimization routine uses a nonlinear gradient method to find the spend levels for our channels that will produce the lowest blended CAC (i.e., total spend / total conversion), given constraints on overall and channel-level budgets (defined by the two b_mt constants and the value of B). Constraints are discussed with marketing stakeholders to ensure optimization is performed under realistic conditions and results are actionable. By comparing the algorithmic recommendations to historical spending patterns we can identify new channel development opportunities that might go unnoticed when looking at a single channel in isolation.

優化程序使用非線性梯度方法來查找我們的渠道的支出水平，該水平將產生最低的混合 CAC(即總支出/總轉化 )，這要考慮到總體預算和渠道預算(由兩個b_mt常數和B的值)。 與市場利益相關者討論約束條件，以確保在現實條件下執行優化，并且結果是可行的。 通過將算法建議與歷史支出模式進行比較，我們可以確定在單獨查看單個渠道時可能不會注意到的新渠道開發機會。

結論 (Conclusion)

Media mix modeling is a powerful tool for measuring and managing a complex marketing mix. By accounting for marketing spend saturation, advertising decay, and the marketing funnel hierarchy, the MMM offers a flexible tool for evaluating the performance of both online and offline marketing channels. Moreover, using a Bayesian framework provides us with the ability to incorporate existing marketing knowledge into the model. The MMM will continue to evolve in form and function as it is calibrated based on lift tests of various marketing channels. In the meantime, we will also keep exploring other areas where this model can be helpful, such as demand forecasting and business planning.

媒體組合建模是衡量和管理復雜營銷組合的強大工具。 通過考慮營銷支出飽和度，廣告衰減和營銷渠道層次結構，MMM提供了一種靈活的工具來評估在線和離線營銷渠道的效果。 此外，使用貝葉斯框架使我們能夠將現有的營銷知識整合到模型中。 MMM將根據各種營銷渠道的提升測試進行校準，因此其形式和功能將繼續發展。 同時，我們還將繼續探索該模型可能有幫助的其他領域，例如需求預測和業務計劃。

[1] Borden, N. H. (1964). The concept of the marketing mix. Journal of advertising research, 4, 2–7.

[1] Borden，NH(1964)。 營銷組合的概念 。 廣告研究雜志，4，2–7 。

[2] Jin, Y., Wang, W., Sun, Y., Chan, D., & Koehler, J. (2017). Bayesian methods for media mix modeling with carryover and shape effects. Google Research.

[2] Jin，Y.，Wang，W.，Sun，Y.，Chan，D.，＆Koehler，J.(2017)。 具有殘留和形狀效應的貝葉斯媒體混合建模方法 。 Google研究。

[3] Juneau, T. (2020). Digital Marketing In a Cookie Less Internet. Forbes.

[3]朱諾，T。(2020)。 少Cookie的Internet中的數字營銷 。 福布斯

[4] Chan, D., Perry, M. (2017) Challenges And Opportunities In Media Mix Modeling, Google Research.

[4] Chan D.，Perry，M.(2017) 媒體混合建模的挑戰與機遇 ，谷歌研究。