In this notebook, we shall use this dataset containing data about passengers from the Titanic. Based on this data, we will use a Ridge Regression model which just means a Logistic Regression model that uses L2 Regularization for predicting whether a person survived the sinking based on their passenger class, sex, the number of their siblings/spouses aboard, the number of their parents/children aboard and the fare they payed.

在這款筆記本上，我們將使用這種含有大約從泰坦尼克號的乘客數據集。 基于此數據，我們將使用Ridge回歸模型，該模型僅表示使用L2正則化的Logistic回歸模型，根據乘客的等級，性別，同胞/配偶的人數，人數來預測某人是否沉沒。他們的父母/孩子和他們支付的車費。

First, we import everything we need for plotting data and creating a great model to make predictions on the data.

首先，我們導入繪制數據和創建出色模型以對數據進行預測所需的一切。

數據探索 (Data Exploration)

Here, we can see what the data actually looks like. The first column indicates whether the person survived with a 1 or a 0, where the 1 stands for survival and the 0 for death. The rest of the columns are all our input columns used to predict the survival. We will, however, forget about as it does not hold important information needed to predict survival. You can also see below that we have 887 persons with their data and 8 total columns where 6 of them will be the input values and 1 of them (the Survived column) the corresponding label.

在這里，我們可以看到數據的實際外觀。 第一列表示該人是生存還是生存1或0，其中1代表生存，0代表死亡。 其余的列都是我們用來預測生存率的所有輸入列。 但是，我們將忘記它，因為它不具備預測生存所需的重要信息。 您還可以在下面看到，我們有887個人及其數據，共有8列，其中6個為輸入值，其中1個(生存列)為相應的標簽。

To get a little bit more familiar with the data we can do some computations with it and plot it. First we print how big the part of the survivors is which can also be described as the total probability of survival.

為了更熟悉數據，我們可以對其進行一些計算并繪制出圖表。 首先，我們打印幸存者部分的大小，這也可以描述為生存的總概率。

When we look at how likely different people from different classes and of different sexes are to survive we can see a clear trend that the higher the class the higher the survival probability and also that women are more likely to survive. Ever wondered why is this the case? The answer is quite simple.

當我們觀察來自不同階級和性別的不同人的生存可能性時，我們可以清楚地看到一個趨勢，即階級越高，生存的可能性就越高，而且女性生存的可能性就越大。 有沒有想過為什么會這樣呢？ 答案很簡單。

When the Titanic began to sink, women and children would go off-board in the lifeboats first before the men. The lower class passengers were not treated equally at the time of sinking as there were so many people in the lower class that not all could not be informed by the stewardesses. Subsequently, it took much longer for them to get to the deck for rescue while first and second class passengers were already boarding the lifeboats. Also, the sailors fastened down the hatchways leading to the third-class section. They said they wanted to keep the air down there so the vessel could stay up longer. It meant all hope was gone for the passengers still down there.

當泰坦尼克號沉沒時，婦女和兒童將首先在救生艇上駛離船上，然后才駛向男子。 在下沉時，下層旅客沒有得到同等的對待，因為下層旅客太多，因此乘務員并不能告知所有人。 隨后，當頭等艙和二等艙乘客已經登上救生艇時，他們花了更長的時間才能到達甲板上進行救援。 此外，水手們將艙口固定在通往三等艙的艙口上。 他們說，他們希望將空氣保持在那里，以便船只可以停留更長的時間。 這意味著所有仍在那里的乘客的希望已蕩然無存。

Another reason why so many people died was the missing safety measures onboard the Titanic. For example, there were not enough boats for the passengers to escape the ship. The lifeboats would have only been sufficient for half the people onboard and due to bad organization not all of them were completely filled. More than half of the passengers were left behind. One good aspect, however, is that the laws for a ship’s safety have become more strict since this disaster. If you want to read about the sinking in detail, have a look at this: https://en.wikipedia.org/wiki/Sinking_of_the_RMS_Titanic

如此之多的人死亡的另一個原因是泰坦尼克號上缺少安全措施。 例如，沒有足夠的船只供乘客逃脫。 救生艇僅能滿足船上一半人的需要，而且由于組織不善，并非所有人都被完全裝滿。 一半以上的乘客被拋在后面。 但是，一個好的方面是，自這場災難以來，船舶安全的法律變得更加嚴格。 如果您想詳細了解下沉，請查看以下內容： https : //en.wikipedia.org/wiki/Sinking_of_the_RMS_Titanic

Looking at the prices, which are all measured in pounds, we can see the total average fare and then the different ones from the different classes. Note that due to inflation these numbers measured in pounds today would be a lot higher.

查看全部以磅為單位的價格，我們可以看到總平均票價，然后是不同類別的票價。 請注意，由于通貨膨脹，今天這些以磅為單位的數字會高得多。

There are also people from all ages on board while the average age is 30 years.

船上還有各個年齡段的人，平均年齡為30歲。

To see the difference of survival probability already mentioned and explained above more visually, we can plot them like the following plots show. Here, you can see the difference between the different classes and sex very well.

為了更直觀地看到上面已經提到和解釋的生存概率的差異，我們可以像下面的圖所示繪制它們。 在這里，您可以很好地看到不同階級和性別之間的差異。

Let’s now look at the fare distribution and the costs from the different classes.

現在，讓我們看一下不同類別的票價分配和費用。

In the following we can clearly see that most passengers did not have any siblings/spouses or parents/children aboard.

在下面的內容中，我們可以清楚地看到，大多數乘客沒有兄弟姐妹/配偶，也沒有父母/子女。

Lastly we look at the distribution of the ages of the passengers.

最后，我們看看乘客的年齡分布。

資料準備 (Data Preparation)

Now to be able to train our model we want to convert our pandas dataframe into PyTorch Tensors. To do this we define the dataframe_to_arrays method which does the conversion to NumPy arrays. To use the function we need to specify 3 kinds of columns namely input columns, categorical columns (columns that do not contain numbers but rather a string standing for a category) and output columns so that it can properly create a NumPy array for input data and labels with all input data from the input columns (by first converting the categorical columns to numerical ones) and the labels from the output columns. Then we can easily convert them to PyTorch Tensors and specify the desired data types so that we are ready to define the model to be ready for the training on the data. Note also that the normalize parameter is set to True which makes the function normalize the input data by squishing all values in a range between 0 and 1 with Min Max Normalization for the model to be able to better learn from the data as it is more uniform now.

現在要能夠訓練我們的模型，我們想將熊貓數據框轉換為PyTorch張量。 為此，我們定義了dataframe_to_arrays方法，該方法可以轉換為NumPy數組。 要使用該函數，我們需要指定3種列，即輸入列，分類列(不包含數字，而是代表類別的字符串的列)和輸出列，以便它可以正確地為輸入數據和輸入創建NumPy數組。標簽，其中包含來自輸??入列的所有輸入數據(首先將分類列轉換為數字列)，以及來自輸出列的標簽。 然后，我們可以輕松地將它們轉換為PyTorch張量并指定所需的數據類型，以便我們準備定義模型以進行數據訓練。 另請注意，將normalize參數設置為True ，這會使函數通過使用Min Max Normalization擠壓0到1之間范圍內的所有值來對輸入數據進行歸一化，以便模型可以更好地從數據中學習，因為它更統一現在。

Now that we have the PyTorch Tensors for the input data and the labels we put them into a PyTorch TensorDataset which contains pairs of inputs and labels.

現在我們有了用于輸入數據和標簽的PyTorch張量，我們將它們放入包含輸入和標簽對的PyTorch TensorDataset中。

Another thing we have to do is to split the original dataset into one for training the model and another one for validating that the model is learning something. This means that the validation dataset contains data that the model has never seen before and by making predictions on it we can see how well the model can perform on unknown data. This accuracy from the validation data will be used as a metric for all training epochs as well as the loss on the validation data.

我們要做的另一件事是將原始數據集分為一個用于訓練模型的數據集，另一個用于驗證模型正在學習某些東西的數據集。 這意味著驗證數據集包含模型從未見過的數據，并且通過對其進行預測，我們可以看到模型對未知數據的性能如何。 驗證數據的準確性將用作所有訓練時期以及驗證數據損失的度量。

Last thing we do with our data is to put it into a DataLoader (one for the validation data and one for the training data) which will be used to train the model with shuffled and batched data.

我們對數據做的最后一件事是將其放入DataLoader (一個用于驗證數據，另一個用于訓練數據)，該數據將用于使用混排和批處理的數據來訓練模型。

定義模型結構 (Defining Model Structure)

Now we can create our model which is just a simple Logistic Regression model with a linear layer that accepts 6 inputs and outputs 1 value between 0 and 1 which basically makes the model’s forward method return the probability of survival it predicts by using the sigmoid activation function. This is necessary as then we can train the model to output a 1 when it thinks that the person would survive and a 0 when it does think that the person will not survive even though the model will probably never return a 1 or a 0 but it will predict a probability closer to 1 or 0 after some time of training. Moreover we define some other methods in our model for training and computing accuracies or printing them. One more thing to note is that as a loss function in training_step we use the Binary Cross Entropy loss. Lastly, we create an instance of the TitanicModel called model which we will train on the training data.

現在我們可以創建我們的模型，它只是一個簡單的Logistic回歸模型，該模型具有一個線性層，該層可以接受6個輸入并輸出0到1之間的1值，這基本上使該模型的forward方法返回使用S形激活函數預測的生存概率。 這是必要的，因為這樣我們可以訓練模型，當模型認為某人可以生存時輸出1，而當模型認為該人可能永遠不會返回1或0時卻認為該人將無法生存，則輸出0。經過一段時間的訓練，將預測接近1或0的概率。 此外，我們在模型中定義了一些其他方法來訓練和計算精度或打印它們。 還有一點要注意的是，作為training_step的損失函數，我們使用了二進制交叉熵損失。 最后，我們創建稱為model的TitanicModel實例，我們將在訓練數據上進行訓練。

訓練模型 (Training the Model)

Now comes the cool part — the actual training! For this we need to make a fit_one_cycle function to do the training for our model. Like any usual fit functions this one uses an optimizer to adjust the models parameters with a certain learning rate according to the gradients (the gradients are just the partial derivatives for each parameter with respect to the loss) which are obtained by backpropagating the loss backwards through the model. Here however there are some things about the fit function I want to point out that are not just the usual computing loss and then adjusting weights and biases thing.

現在是最酷的部分-實際訓練！ 為此，我們需要創建一個fit_one_cycle函數來對我們的模型進行訓練。 像任何通常的擬合函數一樣，該函數使用優化器根據梯度(梯度只是每個參數相對于損耗的偏導數)以一定的學習率調整模型參數，這些梯度是通過反向傳播損耗而獲得的。該模型。 但是，在這里我要指出一些關于擬合函數的問題，這些事情不僅是通常的計算損失，而且還有調整權重和偏差的事情。

• Learning rate scheduling: This is a technique replacing the fixed learning rate usually done by changing the learning rate after every batch of training. This can be done several ways but the way we will do it is with the “One Cycle Learning Policy” which starts with a smaller learning rate and then starts to gradually increase for the first 30% of epochs and then decreasing it again for optimal learning. For this scheduler we just need to set the maximum learning rate to which it will increase over time. If you want to go deeper into this topic, I suggest you read this: https://sgugger.github.io/the-1cycle-policy.html

  學習率安排 ：這是一種通常通過在每批培訓后更改學習率來代替固定學習率的技術。 這可以通過多種方式完成，但是我們將采用“單周期學習策略”來實現 ，該策略以較小的學習率開始，然后在前30％的時期開始逐漸增加，然后再次降低以實現最佳學習。 對于此調度程序，我們只需要設置隨時間增加的最大學習率即可。 如果您想更深入地研究這個主題，建議您閱讀以下內容： https : //sgugger.github.io/the-1cycle-policy.html

• Weight decay / L2 Regularization: Another thing we use is weight decay which adds the sum of the weights squared to the loss function so that bigger weights will be punished as bigger weights are usually a sign of overfitting. Thereby we make the model able to generalize better and achieve better results on unknown data as the weights are being lowered. See this for more information about weight decay: https://towardsdatascience.com/this-thing-called-weight-decay-a7cd4bcfccab

  權重衰減/ L2正則化 ：我們使用的另一種方法是權重衰減，它會將權重的平方和加到損失函數上，以便懲罰更大的權重，因為更大的權重通常表示過度擬合。 因此，隨著權重的降低，我們使模型能夠更好地泛化并在未知數據上獲得更好的結果。 請參閱此以獲取有關體重減輕的更多信息： https : //towardsdatascience.com/this-thing-叫做-weight-decay- a7cd4bcfccab

• Gradient clipping: Lastly there is gradient clipping. This is actually quite simple but still very useful. The way gradient clipping works is that it just limits the gradient to a certain value so that if the gradient would take the model in the wrong direction it is limited to a certain size which means that the model can’t be hurt due to large gradient values. Here is an interesting post about it: https://towardsdatascience.com/what-is-gradient-clipping-b8e815cdfb48

  漸變裁剪 ：最后是漸變裁剪。 這實際上很簡單，但仍然非常有用。 漸變裁剪的工作方式是將漸變限制為某個值，因此，如果漸變將模型沿錯誤的方向放置，則將其限制為一定的大小，這意味著不會因大漸變而傷害模型價值觀。 這是關于它的一個有趣的帖子： https : //towardsdatascience.com/what-is-gradient-clipping-b8e815cdfb48

Now, we should be all set to start the training. First, we compute the accuracy and loss on the validation data to see how good the model initially performed vs how it performs after the training. For the training itself we define the hyperparameters like maximum learning rate, epochs to train for, weigh decay value, gradient clipping and the optimizer which will be the adam optimizer.

現在，我們都應該開始培訓了。 首先，我們計算驗證數據的準確性和損失，以查看模型最初執行的效果與訓練后的執行效果。 對于訓練本身，我們定義了超參數，例如最大學習率，要訓練的時期，權重衰減值，梯度削波以及將作為亞當優化器的優化器。

After this brief training, our model should be very good at predicting survival for Titanic passengers (Note that it can never be perfect as there is always a component of luck involved in the individual probability for survival). You can also see this when you look at the accuracy and the loss on the validation data and compare it to the one computed before the training. To verify this further, let’s plot the accuracies on the validation data over the course of the training phase as well as the loss on both training data and validation data. This last thing can also show us how much the model is overfitting since as soon as the training data loss decreases, the validation data loss increases or stays the same. We are overfitting since we are getting better and better on the training data, but worse on the validation data which is definitely not what we want. However, the model does not seem to be overfitting, which is great!

經過簡短的培訓后，我們的模型應該非常擅長預測泰坦尼克號乘客的生存(注意，由于運氣總是與個人生存概率有關，因此它永遠不可能是完美的)。 當您查看驗證數據的準確性和損失并將其與訓練之前計算出的值進行比較時，也可以看到這一點。 為了進一步驗證這一點，讓我們在訓練階段的過程中繪制驗證數據的準確性，以及訓練數據和驗證數據的損失。 這最后一件事還可以向我們顯示模型有多適合，因為一旦訓練數據丟失減少，驗證數據丟失就會增加或保持不變。 我們正在過度擬合，因為我們在訓練數據上越來越好，但在驗證數據上卻越來越差，這絕對不是我們想要的。 但是，該模型似乎并不過分擬合，這太棒了！

保存模型 (Save the Model)

Now we need to save our model by writing its state which means all of its parameters to a file and log our hyperparameters, final accuracy and final loss so we can later easily compare different model architectures and different choices of hyperparameters to see how well they perform.

現在我們需要通過寫入模型的狀態來保存模型，這意味著將其所有參數寫入文件并記錄我們的超參數，最終精度和最終損失，以便稍后可以輕松比較不同的模型架構和不同的超參數選擇，以查看它們的性能如何。

Note that when looking at the weights from the model.state_dict() output we can see how important each of the input values is. For example, we can see that the class is associated with a negative value which is good since people from a class described with a higher number like class 3 were less likely to survive. The next weight shows the extreme importance of the sex for prediction as it is associated with the largest negative value which can be understood, if you know that a man is represented with 1 and a woman with 0. What we can also deduce from the last weight is that the larger the fare paid the higher the survival probability which makes sense, too.

請注意，當查看來自model.state_dict()輸出的權重時，我們可以看到每個輸入值的重要性。 例如，我們可以看到該類別與一個負值相關聯，這是一個很好的值，因為來自該類別的人們用較高的數字(例如第3類)描述的生存可能性較小。 下一個權重顯示了性別對于預測的極端重要性，因為它與可以理解的最大負值相關，如果您知道男人用1表示，女人用0表示。我們也可以從上一個推論得出權重是，票價越高，生存率越高，這也是有道理的。

在樣品上測試模型 (Test the Model on Samples)

Having the training phase behind us, we can do some testing on various single examples from the validation data to get a feeling for how well the model performs. Therefore, we need to make a function which will return the models prediction for a given dataset element as well as what the person’s data is and whether the person actually survived or not. As you can see in order to display the data it is important that we denormalize our data again by putting all values from the range between 0 and 1 back to their initial range and converting the categorical column Sex back to the strings female and male from the numbers 0 and 1 as which they were represented in the dataset.

有了培訓階段，我們可以對來自驗證數據的各個示例進行一些測試，以了解模型的性能。 因此，我們需要創建一個函數，該函數將返回給定數據集元素的模型預測，以及該人的數據是什么，以及該人是否實際幸存。 如您所見，為了顯示數據，重要的是我們再次對數據進行非規范化，方法是將0到1之間的所有值都放回其初始范圍，然后將分類列Sex轉換為從在數據集中表示的數字0和1。

As expected, the model gets most predictions right with a survival probability that makes complete sense when looking at the input data. Even though it was wrong on the first prediction, this is not a bad sign since there is also always a component of luck involved which makes a case like this not perfectly predictable. If we recall the survival probabilities for the persons of different sexes and of different classes, we can see that the prediction is actually pretty close to that, which I think is a good sign.

正如預期的那樣，該模型可以通過查看輸入數據時完全有意義的生存概率正確地進行大多數預測。 即使在第一個預測中是錯誤的，但這也不是一個壞兆頭，因為總會有涉及運氣的因素，這使得這種情況無法完全預測。 如果我們回想起不同性別和不同階級的人的生存概率，我們可以看到預測實際上與該預測非常接近，我認為這是一個好兆頭。

Don’t you want to find out as well whether you would have survived the Titanic disaster. To do this we have a nice function that asks you to input your data and then returns its prediction after converting the categorical values to numericals and normalizing the input data. Just think of a fare reasonable for your chosen class (or not and try to break the predictions). You can, of course, completely make up data to test the model and see which people would have survived.

您是否也不想找出是否可以在泰坦尼克號災難中幸存下來。 為此，我們有一個很好的函數，要求您輸入數據，然后在將分類值轉換為數值并對輸入數據進行歸一化后返回其預測。 只需考慮適合您所選班級的票價(否則，請嘗試打破預期)。 當然，您可以完全組成數據來測試模型，并查看哪些人可以幸免。

Lastly we can make a submission .csv file for the Titanic competition on kaggle to become first place p ;).

最后，我們可以為kaggle上的泰坦尼克號比賽提交.csv文件，以成為第一名p;)。

See the entire notebook here.

在這里查看整個筆記本。

總結和未來工作的機會 (Summary and Opportunities for future work)

Lastly, I want to summarize the amazing things I learned from this nice project at Jovian. The first major takeaway was how to deal with Pandas dataframes and data in general which was usually done for me when I was provided a starter notebook. Now that I did this project from scratch I read about the pandas library and its various functions so I was able to use this data for my project very well. I also learned quite a bit about data normalization.

最后，我想總結一下我從Jovian的一個不錯的項目中學到的令人驚奇的事情。 第一個主要的收獲是如何處理Pandas數據框和一般數據，這在為我提供入門筆記本時通常為我完成。 現在，我從頭開始做這個項目，我了解了pandas庫及其各種功能，因此我能夠很好地將這些數據用于我的項目。 我還學到了很多有關數據標準化的知識。

Another thing I took away from this was a lot of knowledge about Logistic Regression as I read quite a lot on the various approaches. For example, I read about why you would use 1 output neuron vs 2 output neurons for binary classification and came to the result that the usage of 1 output neuron is less prone to overfitting as it has less parameters which makes totally sense. This is also why I used this for my model with the Binary Cross Entropy. Moreover, I learned the math behind regularization to be able to better understand it and implement it which helped a lot when implementing regularization and choosing the weight decay hyperparameter.

我從中學到的另一件事是關于邏輯回歸的大量知識，因為我對各種方法學到了很多東西。 例如，我讀到了為什么要使用1個輸出神經元而不是2個輸出神經元進行二進制分類，并且得出的結果是，使用1個輸出神經元具有較少的參數就不太容易過擬合，因為這完全有意義。 這也是為什么我將其用于具有二進制交叉熵的模型的原因。 此外，我了解了正則化背后的數學知識，以便能夠更好地理解它并實現它，這在實現正則化和選擇權重衰減超參數時很有幫助。

Not to forget are also the things I learned about the disaster by examining the data and also from additional research, which was very interesting.

我也不要忘記通過檢查數據以及從其他研究中學到的有關災難的知識，這非常有趣。

To sum up, I cannot stress enough on how great such projects are for learning as by doing everything yourself you can learn much better. I feel more comfortable with the PyTorch library and Machine Learning now.

綜上所述，我不能過多強調此類項目對學習的意義，因為您自己做的一切都可以使您學得更好。 現在，我對PyTorch庫和機器學習感到更加自在。

I can’t wait to work on more challenging projects in future with other datasets and compete in various interesting Kaggle challenges with all the newly-learned things to deepen my knowledge in the world of AI and have fun. I am really looking forward to doing the thousands of projects that I have in my mind!

我迫不及待地希望將來與其他數據集一起從事更具挑戰性的項目，并與所有新近學習的知識一起競爭各種有趣的Kaggle挑戰，以加深我在AI世界中的知識并從中獲得樂趣。 我真的很想做我腦海中成千上萬的項目！