The bias-variance tradeoff is one of the most important but overlooked and misunderstood topics in ML. So, here we want to cover the topic in a simple and short way as possible.

偏差-方差折衷是機器學習中最重要但被忽視和誤解的主題之一。 因此，在這里我們想以一種簡單而簡短的方式涵蓋這個主題。

Let’s start with basics and see why it is important and how this concept is to be used. We want to keep this crisp so we’ll talk in pointers at times. By the end of this, you would know:

讓我們從基礎開始，看看為什么它很重要以及如何使用此概念。 我們希望保持清晰，因此我們有時會談指針。 到此為止，您將知道：

• Bias
  偏壓
• Variance
  方差
• Their relationship
  他們的關系
• importance of their tradeoff
  權衡的重要性
• how to analyze the model condition and take necessary steps
  如何分析模型條件并采取必要步驟

So what this Bias- Variance tradeoff exactly has to do with performance?

那么，這種偏差與偏差的權衡與性能究竟有什么關系呢？

You build a model. The model doesn’t perform well. You want to improve the performance but don’t know where to start.

您建立模型。 該模型的效果不佳。 您想提高性能，但不知道從哪里開始。

A diagnosis is important as it pin-points the areas of improvement. You need to clearly identify the components which are leading to a poor model.

診斷很重要，因為它可以指出需要改進的地方。 您需要清楚地識別導致模型不良的組件。

Issue: Bad model performance

問題 ：模型性能不佳

Focus area for the fix: Prediction error

修復的重點領域：預測錯誤

Before jumping to the topic, just know this.

在跳到主題之前，請先知道這一點。

Total Error = Bias^2 + Variance + Irreducible error

Total Error = Bias^2 + Variance + Irreducible error

a. Total error = Prediction error that we are trying to minimize

一個。 總誤差 =我們試圖最小化的預測誤差

b. Bias error = Difference between the average prediction of model and the correct prediction

b。 偏差誤差 =模型的平均預測與正確預測之間的差異

c. Variance = Variability of a model prediction for a given data point (difference in results for the same data point if training data is changed).

C。 方差 =給定數據點的模型預測的方差 (如果更改訓練數據，則同一數據點的結果差異)。

d. Irreducible error = It is the inherent error of data which is caused by the distribution of data and other specification. It is just the way the data is and basically nothing can be done about it.

d。 不可減少的錯誤 =這是由數據的分布和其他規范引起的數據固有的錯誤。 這只是數據的方式，基本上它無法做任何事情。

Okay, these are formal definitions. How to visualize them and understand them in normal terms.

好的，這些是正式的定義。 如何可視化它們并以常規術語理解它們。

Goal — Low Bias, Low Variance

目標—低偏見，低方差

Let’s see each of the possible combinations and understand each of them practically with the above representation.

讓我們看一下每種可能的組合，并通過上述表示實際理解它們。

a. High Bias, high Variance: Worst Case- Results not close to the target(High Bias) and not even consistent in any direction(High variance).

一個。 高偏差，高方差： 最壞的情況-結果不接近目標(高偏差)，甚至在任何方向上都不一致(高偏差)。

b. High Bias, low variance: Results not close to the target (High Bias) but consistent in one direction(Low Variance).

b。 高偏差，低方差：結果不接近目標(高偏差)，但在一個方向上一致(低方差)。

c. Low Bias, high Variance: Results close to the target (Low Bias) but not consistent around the target(High Variance).

C。 低偏差，高方差：結果接近目標(低偏差)，但在目標周圍(高方差)不一致。

d.Low Bias, low variance: Best Case- Results close to the target (Low Bias) and consistent around the target(Low Variance).

d。 低偏差，低方差： 最佳情況-結果接近目標(低偏差)，并且在目標附近(低方差)保持一致。

Now the question is why it is a tradeoff. Why not simply go and get low bias low variance. This is because of the way bias and variance are related, each comes at the cost of other. When you try to improve one, the other gets worse. Like if you cook on low flame, it takes forever. You increase the flame, food starts burning. You have to find a point where both are balanced.

現在的問題是，為什么要進行權衡。 為什么不簡單地去獲得低偏差低方差。 這是因為偏差和方差之間是相關的，每個都以其他為代價。 當您嘗試改善一個時，另一個會變得更糟。 就像您在低火上烹飪一樣，它需要永遠。 您增加火焰，食物開始燃燒。 您必須找到一個平衡點。

Ideal model: Learns the underlying patterns in training data just optimally and creates a generalized algorithm that can work with similar unseen data as well.

理想模型：以最佳方式學習訓練數據中的基礎模式，并創建一種通用算法，該算法也可以處理相似的看不見的數據。

Overfitting: The model makes a very highly fitting algorithm tailored for the training data specifically. Thus, it cannot stand variations that come with unseen data.

過度擬合：模型針對訓練數據制定了非常適合的算法。 因此，它無法忍受看不見的數據帶來的變化。

An overfitting model can be understood as a “Frog in the well” who became too comfortable in the present scenario(training data) but its present understanding won’t help to survive a different surrounding(test data).

過度擬合模型可以理解為“井中的青蛙”，他在當前場景(訓練數據)中變得太自在了，但其目前的理解無助于在不同的環境中生存(測試數據)。

Underfitting: The model makes a very loose-fitting algorithm that can’t even work for the training data as it couldn’t learn the patterns as it oversimplified everything. Thus it cannot give correct answers.

欠擬合：模型提出了一種非常寬松的算法，該算法甚至不能用于訓練數據，因為它過于簡化了所有操作，因此無法學習模式。 因此，它不能給出正確的答案。

An underfitting model is a person who thinks he learned a skill by just taking the intro session and learning buzz words or he became a cricket player just because he knows how to hit a ball.

不稱職的模特是一個人，他認為自己只是通過參加入門課程并學習流行語來學習技能，或者僅僅因為他知道如何擊球而成為板球運動員。

You can read the detailed explanation below:

您可以閱讀以下詳細說明：

https://medium.com/analytics-vidhya/understanding-how-machine-learning-is-just-like-the-human-learning-process-801a0bca3e56

https://medium.com/analytics-vidhya/understanding-how-machine-learning-is-just-like-the-human-learning-process-801a0bca3e56

The goal was to build a model that gives-

目的是建立一個模型，使-

• Right results most of the times.

  在大多數情況下，正確的結果。

Models with Overfitting have high variance and ones with Underfitting have a high bias.

具有過度擬合的模型具有較高的方差，而具有欠擬合的模型具有較高的偏差。

What do I keep in mind regarding these to solve them in real-time?

我要牢記這些以實時解決這些問題？

• Identify whether your model suffers from overfitting or underfitting. Use the train-test accuracy of the model for this.
  確定您的模型是過度擬合還是擬合不足。 為此，請使用模型的訓練測試精度。

• Take measures as follows once the issue is identified.
  一旦發現問題，請采取以下措施。

a. Problem: High Variance(This will be solved the way overfitting is solved)

一個。 問題：高方差(將通過解決過度擬合的方式解決)

Let’s see each solution and how exactly it is solving the issue.

讓我們看看每種解決方案以及它如何解決問題。

• Add more training data: You have learned very data specific. Here’s more data for increasing your general understanding so that it is no longer data specific.

  添加更多培訓數據 ：您已經學到了非常具體的數據。 這里有更多數據可用于增強您的一般理解，從而不再是特定于數據的數據。

• Data augmentation: I don’t have much data. Let me modify current data to create more variations and present them to you for your better understanding.

  數據擴充 ：我沒有太多數據。 讓我修改當前數據以創建更多變體，然后將其呈現給您，以使您更好地理解。

• Reduce the complexity of the model: You have learned unnecessary stuff. These specific details are not required. Retain only what can be applied everywhere and let go of rest to simplify.

  降低模型的復雜性 ：您已經學到了不必要的東西。 這些特定的細節不是必需的。 僅保留可以在任何地方應用的內容，并放手休息以簡化操作。

• Bagging(stands for Bootstrap Aggregating): You are giving different answers every time I change the training data a little. Let me randomly sample the data and give to you all the samples. You create predictors and train on each sample and get all the different results you can. Put together all learning by aggregating all the results and give me one final answer which will remain consistent.

  Bagging(代表B ootstrap Agg的注冊) ：每次我稍微改變訓練數據，您都會給出不同的答案。 讓我隨機抽樣數據，然后給您所有樣本。 您可以創建預測變量并對每個樣本進行訓練，并獲得所有不同的結果。 通過匯總所有結果來匯總所有學習內容，并給我一個最終答案，它將保持一致。

Note: The different predictors need to have minimum correlation so that they make “different errors”(not to be confused with the model, we have 1 model having different predictors that gives results of different samples).

注意：不同的預測變量需要具有最小的相關性，以使它們產生“不同的誤差”(不要與模型混淆，我們有1個具有不同預測變量的模型可以得出不同樣本的結果)。

b. Problem: High Bias(This will be solved the way underfitting is solved).

b。 問題 ：高偏差(這將通過解決欠擬合的方式解決)。

• Add features: You gave a result that Person A won’t be able to repay the loan because he is old(feature). You are saying this because an old Person B couldn’t repay it. But you also need to see their annual income, past history, etc(other features) and then decide.

  添加功能：您得出的結果是，人A由于年齡大(功能)而無法償還貸款。 您之所以這樣說，是因為老人B無法償還。 但是，您還需要查看他們的年收入，過去的歷史記錄等(其他功能)，然后做出決定。

• Use a model with higher complexity: We need to replace you with someone who can understand the relation between different parts of data and how they work together better than you.

  使用具有更高復雜性的模型：我們需要以能夠理解數據不同部分之間的關??系以及它們如何更好地協同工作的人來代替您。

• Boosting: I don’t trust you. You create predictors and ask them each to answer. We’ll ask each predictor about the logic they used to get their partially right answers. Whenever we get some part right, we’ll add that logic in the rule. Each one will have their shortcoming but together, they will cover up for each other. They’ll work as a team to finally create a well-fitting complex rule.

  提振：我不相信你。 您創建預測變量，并要求它們各自回答。 我們將詢問每個預測變量有關他們用來獲得部分正確答案的邏輯。 每當我們得到正確的部分時，我們都會在規則中添加該邏輯。 每個人都會有自己的缺點，但是他們會互相掩蓋。 他們將作為一個團隊工作，以最終創建一個合適的復雜規則。

Note: The team of weak learners should have a minimum correlation between them, otherwise everyone would have the right answers for the same sections and some sections will be left answered incorrectly.

注意：弱學習者團隊之間的相關性應最低，否則每個人對相同部分的回答都是正確的，而某些部分的回答將不正確。

Hope this helped to understand the topic and gave the understanding to leverage the concept as well.

希望這有助于理解該主題，并給予理解以利用該概念。

Let us know your feedback. Thanks for reading!

讓我們知道您的反饋。 謝謝閱讀！

Sources:

資料來源：

Fig 1, Fig 2: http://scott.fortmann-roe.com/docs/BiasVariance.html

圖1，圖2： http ： //scott.fortmann-roe.com/docs/BiasVariance.htm