In the first part of my series on experimental design Thinking About Experimental Design, we covered the foundations of an experiment: the goals, the conditions, and the metrics. In this post, we will move away from the initial experimental set up to begin understanding baseline metrics and the nuances of picking appropriate conversion metrics and rates.

在我的實驗設計系列的第一部分中，“ 思考實驗設計”中 ，我們介紹了實驗的基礎：目標，條件和度量。 在這篇文章中，我們將脫離最初的實驗設置，開始了解基線指標以及選擇合適的轉化指標和費率的細微差別。

介紹 (Introduction)

I introduced the goals of an experiment a business context, using the example of a lemonade stand: measuring the difference in outcomes (# of cups sold) under controlled conditions (color of cup) during comparable time-frames. In the example, I failed to mention a critical element of any experiment — a hypothesis, a preliminary presumption based on limited evidence. Strictly speaking, the goal of an experiment is to validate or refute ideas put forth by a hypothesis. A reasonable hypothesis for our lemonade stand example might be that “the color of the lemonade cup affects the number of cups sold.”

我以檸檬水攤位為例，介紹了在商業環境中進行實驗的目標：在可比較的時間范圍內，在受控條件 (杯子的顏色)下測量結果 (所售杯子的數量)的差異 。 在示例中，我沒有提到任何實驗的關鍵要素- 假設 ，即基于有限證據的初步假設。 嚴格來說，實驗的目的是驗證或駁斥假設提出的想法。 我們的檸檬水攤位示例的合理假設可能是“檸檬水杯的顏色會影響所售杯子的數量”。

從基線開始 (Starting from a Baseline)

If we’re just starting our lemonade stand (or other small business/business line), can you see a problem with immediately designing this particular experiment (or any experiment) to validate a hypothesis? Market research aside, we don’t have any reliable (backed by hard-data) predictions on what to expect for our number of cups sold!

如果我們只是開始我們的檸檬水攤位(或其他小型企業/業務線)，您是否會立即設計此特定實驗(或任何實驗)以驗證假設是否存在問題？ 除了市場研究，我們對售出的杯子數量還沒有可靠的預測(有硬數據支持)！

If we have poor sales the first two weeks with red cups, followed by boosted sales in the next two weeks with blue cups, I wouldn’t be comfortable concluding that blue cups are superior. The sudden change in sales may be due to the fact that people didn’t know about our stand in its opening weeks, and only recently discovered our stand. Remember, the goal of an experiment is to understand the effects of incremental changes. When we introduce a big change (ie. starting our stand or completely revamping our storefront), we introduce instability into our existing business, which will mask and/or distort the effect of our experimental conditions.

如果我們在頭兩周使用紅色杯子的銷售情況不佳，然后在接下來的兩周使用藍色杯子的銷售量有所增長，那么我不能肯定地說藍色杯子是上乘的。 銷售量的突然變化可能是由于人們在開放的幾周內并不了解我們的展位，而只是在最近才發現了我們的展位。 請記住， 實驗的目的是了解增量變化的影響。 當我們進行重大更改時(即啟動我們的展位或完全改造店面)，我們會將不穩定因素引入現有業務，這將掩蓋和/或扭曲我們的實驗條件的影響。

Before conducting an experiment it’s important to establish a stable baseline that will be used the judge the effects our incremental changes.

在進行實驗之前，重要的是要建立一個穩定的基線，該基線將用于判斷我們的增量變化的影響。

To make our example more concrete, let’s say we’ve been running our lemonade stand for 1.5 years. We’re relatively well-known in the community, but we’ve run out of ideas on how we can continue growing our business. After prioritizing our business goals and brainstorming relevant conversion metrics and rates*, we’ve decided to analyze the conversion metric:

為了使我們的示例更具體，假設我們已經運行檸檬水攤位1.5年了。 我們在社區中相對知名，但是關于如何繼續發展業務的想法已經耗盡。 在確定了業務目標的優先級并集體討論了相關的轉化指標和轉化率*之后，我們決定分析轉化指標：

# of cups sold / people who stop by our stand.

售出的杯子數/站在我們展位旁的人。

I have generated some dummy data representing our monthly sales for the past year. The relative stability of our business with respect to the conversion rate. Now, when we introduce our experimental condition (changing the color of the cup), we have a reliably predictable baseline conversion rate (3.62% average) with which we can compare our new outcome.

我已經生成了一些虛擬數據，這些數據代表了過去一年的每月銷售額。 我們的業務相對于轉換率的相對穩定性。 現在，當我們介紹實驗條件(改變杯子的顏色)時，我們有了可靠的可預測的基線轉化率(平均3.62％)，可以與我們比較新的結果。

考慮轉化率的變化 (Thinking about changes in our conversion rate)

At this point, it is easy to forget that our target metric is a conversion rate and begin brainstorming incremental changes that increase # of cups sold. The use of a conversion rate instead of an absolute value requires us to expand our focus from a single metric, to the relationship between two related metrics — from a focus on scale to a focus on scale & efficiency.

在這一點上，很容易忘記我們的目標指標是轉化率，并開始集思廣益，以增加銷售杯數的增量變化。 使用轉換率而非絕對值要求我們將重點從單一指標擴展到兩個相關指標之間的關系-從關注規模到關注規模和效率。

To increase our conversion rate, we must develop a strategy to increase the number of cups sold at a greater rate than we increase the number of people who stop by our stand. Altering the color or design of the cup may be an interesting business initiative; this is assuming that we believe people stop by our stand mostly for the lemonade and our beautiful stand and not due to the color of the cup.

為了提高轉化率，我們必須制定一項戰略，以比增加增加在我們展臺前停留的人數更多的速度增加杯子的銷售數量。 改變杯子的顏色或設計可能是一個有趣的商業嘗試； 這是假設我們相信人們主要是為了檸檬水和漂亮的立場而停下來，而不是因為杯子的顏色。

From a business perspective, we often read about using data and experiments to deliver actionable insights. In addition to our baseline rate, it is important to set reasonable target rates under each of our experimental conditions; we do not want to be taking action on any small change in conversion rates. Setting a target conversion rate is as much an art as it is a science, and can be based on a combination of past data and intuitive business sense. In our lemonade example, we might say that we will switch the color of our cups moving forward if our conversion rate is 4.12% over the next couple of months, an increase of .5%.

從業務角度來看，我們經常閱讀有關使用數據和實驗來提供可行見解的信息。 除了我們的基準速率外，在每個實驗條件下設定合理的目標速率也很重要； 我們不希望對轉化率的任何小變化采取行動。 設定目標轉化率既是一門藝術，也是一門科學，并且可以基于過去的數據和直觀的商業意識相結合。 在我們的檸檬水示例中，我們可以說，如果未來幾個月我們的轉化率為4.12％(增加0.5％)，我們將改變杯子的顏色。

結論： (Conclusion:)

To summarize what we have accomplished in our lemonade example:

總結一下我們在檸檬水示例中所取得的成就：

1. We defined our business goal: increase cups sold

   我們確定了我們的業務目標： 增加杯子銷量

2. We defined our conversion metric and conversion rate: cups sold & cups sold / foot traffic

2.我們定義了轉化指標和轉化率： 售出杯數和售出杯數/人流量

3. We developed a controlled incremental change that will (hypothetically) affect our outcome

3.我們開發了受控的增量更改，該更改將(假設地)影響我們的結果

4. We established a stable baseline for comparison.

4.我們建立了穩定的基線進行比較。

If our conversion metric achieves the target goal, we can conduct business under the new conditions moving forward right? Well, not quite.

如果我們的轉化指標實現了目標，那么我們可以在新的條件下開展業務嗎？ 好吧，不完全是。

So far, you may have realized that I have not introduced any statistics in our experiment! At this point, it may not be entirely clear why we need statistics to validate our hypothesis and complete our experiment. Nonetheless, when dealing with such uncertainty, we want a way of quantifying our decision-making process; we use statistics quantify the strength of our experimental evidence. In the next article of this series, I will begin introducing ideas of basic statistical concepts and test statistics as they apply to our experimental design.

到目前為止，您可能已經意識到我沒有在實驗中引入任何統計信息！ 在這一點上，可能還不清楚，為什么我們需要統計數據來驗證我們的假設并完成我們的實驗。 但是，在處理此類不確定性時，我們需要一種量化決策過程的方法。 我們使用統計數據來量化我們的實驗證據的強度。 在本系列的下一篇文章中，我將開始介紹適用于我們的實驗設計的基本統計概念和測試統計概念。

[1]: Monika Wahi. (2020). The Data Science of Experimental Design. LinkedIn Learning. https://www.linkedin.com/learning/the-data-science-of-experimental-design

[1]：莫妮卡·瓦希(Monika Wahi)。 (2020)。 實驗設計的數據科學。 LinkedIn學習。 https://www.linkedin.com/learning/the-data-science-of-experimental-design

[2]: Icons & Images. Pexels: https://www.pexels.com/

[2]： 圖標和圖像。 Pexels： https ：//www.pexels.com/