聚合數據處理

by Satyam Singh Chauhan

薩蒂揚·辛格·喬漢(Satyam Singh Chauhan)

R中聚合的簡介：強大的數據處理工具 (An introduction to aggregates in R: a powerful tool for playing with data)

Data Visualization is not just about colors and graphs. It’s about exploring the data and visualizing the right thing.

數據可視化不僅涉及顏色和圖形。這是關于探索數據并使可視化正確的事情。

While playing with the data, the most powerful tool that comes handy is Aggregates. Aggregates is just the type of transformation that we apply to any given data.

在處理數據時，最方便使用的最強大的工具是聚合。聚合只是我們應用于任何給定數據的轉換類型。

我們提供11種匯總函數： (We have 11 aggregate function available to us:)

avg
平均

Average of all numeric values is calculated and returned.
計算并返回所有數值的平均值。
count
計數

Function count returns total number of items in each group.
函數計數返回每個組中的項目總數。
first
第一

The first value of each group is returned by the function first.
函數首先返回每個組的第一個值。
last
持續

The last value of each group is returned by the function last.
每個組的最后一個值由函數last返回。
max
最高

The max value of each group is returned by the function max.
每個組的最大值由函數max返回。

It is very helpful to identify outliers as well.
識別異常值也非常有幫助。
median
中位數

The median of all numeric values for the mentioned group is returned by the function median.
函數中位數返回所提及組的所有數值的中位數。
min
分

The min value of each group is returned by the function min.
每個組的最小值由函數min返回。

It is very helpful to identify outliers as well.
識別異常值也非常有幫助。
mode
模式

The mode of all numeric values for the mentioned group is returned by the function mode.
功能組返回所提及組的所有數值的模式。
rms
均方根

Root Mean Square, rms value for all numeric values in the group is returned by the fucntion rms.
均方根均方根值由功能均方根值返回。
sttdev
sttdev

Standard Deviation of all Numeric values given in the group is returned by the function stddev.
函數stddev返回該組中所有給定數值的標準偏差。
sum
和

Sum of all the numeric values is returned by the function sum.
所有數值的總和由函數sum返回。

基本范例 (Basic Examples)

使用聚合函數的基本視覺散點圖-總和 (Basic Visual Scatter plot using aggregate function — sum)

#Include the Librarylibrary(plotly)

#Store the graph in one variable to make it easier to manipulate.p <- plot_ly(     type = 'scatter',     y = iris$Petal.Length/iris$Petal.Width,     x = iris$Species,     mode = 'markers',     marker = list(          size = 15,          color = 'green',          opacity = 0.8     ),     transforms = list(          list(               type = 'aggregate',               groups = iris$Species,               aggregations = list(                    list(                         target = 'y', func = 'sum', enabled = T                    )               )          )     ))

#Display the graphp

這是什么意思？ (What does this mean?)

Function sum, as mentioned above, calculates the sum of each group.Thus, here the groups are categorized as species. This code uses the Iris Data Set which consist three different species, setosa, veriscolor, and virginica. For each species there are 50 observations in the data set. This data set is available in R (built-in) and can be loaded directly.

如上所述，函數sum計算每個組的總和，因此這里將這些組歸類為物種。此代碼使用了鳶尾花數據集，該數據集包含三種不同的樹種：setosa，veriscolor和virginica。對于每個物種，數據集中都有50個觀測值。該數據集位于R(內置)中，可以直接加載。

There are “iris” and “iris3” - two data sets are available. You can choose any one of them to run this code. The Data-Set used in this article is “iris”.

有“ iris”和“ iris3”-兩個數據集可用。您可以選擇其中任何一個來運行此代碼。本文中使用的數據集為“ iris”。

此代碼的作用是什么？ (What does this code do exactly?)

This code uses the function sum and calculates the sum of all the Petal.Length of each group respectively. Then, the calculated sum is plotted on the x-y axis. Where the x-axis is Species, the y-axis shows the Summation.

此代碼使用函數sum并分別計算每個組的所有Petal.Length的總和。然后，將計算出的總和繪制在xy軸上。 x軸為“物種”時，y軸顯示“求和”。

From this graph, we can get an idea that the petal size of setosa is smallest as the sum is the smallest, but it’s not conclusive evidence. To get conclusive evidence we can use the function avg.

從這張圖中，我們可以得出一個結論，setosa的花瓣大小最小，因為總和最小，但這不是決定性的證據 。為了獲得確鑿的證據，我們可以使用函數avg。

The function sum is very suitable for almost the whole data set. For example, one of the best places where this can be used is in Population Data Set. In the world population data set, we can aggregate countries according to continents and find the sum of all the population of the countries in it.

函數和非常適用于幾乎整個數據集 。例如，可以使用的最佳位置之一是“人口數據集”。在世界人口數據集中，我們可以按大洲匯總國家/地區，并找到其中所有國家的總和。

最常用的功能-平均 (Most used function — avg)

#Include the Librarylibrary(plotly)

#Store the graph in one variable to make it easier to manipulate.q <- plot_ly(     type = 'bar',     y = iris$Petal.Length/iris$Petal.Width,     x = iris$Species,     color = iris$Species,     transforms = list(          list(               type = 'aggregate',               groups = iris$Species,               aggregations = list(                    list(                         target = 'y', func = 'avg', enabled = T                    )               )          )     ))

#Display the graphq

這是什么意思？ (What does this mean?)

The iris data-set contains two columns for Petals, Petal.Width and Petal.Length. Further, it can be used to calculate the average of the ratio of Petal.Length & Petal.Width.

虹膜數據集包含用于花瓣的兩列，花瓣寬度和花瓣長度。此外，它可用于計算Petal.Length和Petal.Width之比的平均值。

該代碼的作用是什么？ (What does this code do exactly?)

For each observation, the ratio of Petal.Length to Petal.Width is calculated before the average of all the gained values is plotted. As we can observe from this Bar Plot, Setosa has the max ratio with a near-ratio of 7, which shows that the petal length in Setosa is 7 times longer than its width. While on the other hand, virginica has the smallest ratio with nearly 3 times the width.

對于每個觀察，在繪制所有獲得值的平均值之前，先計算Petal.Length與Petal.Width的比率。從該條形圖中可以看出，Setosa的最大比例接近7，表明Setosa的花瓣長度是其寬度的7倍。另一方面，維吉尼亞具有最小的比例，幾乎是寬度的3倍。

This function is very flexible and especially when it’s used very wisely to get the best result. For example, if we consider some other data-set like Population, then we can calculate the average birth to death ratio for each country.

此功能非常靈活，尤其是在非常明智地使用以獲得最佳效果時。例如，如果我們考慮其他數據集，例如人口，那么我們可以計算每個國家的平均出生與死亡比率。

Let’s use all the functions in one graph. Now we’re going to plot a scatter plot for each category and we’re going to use all the functions. To this graph we will add a button from which we can select the desired function to make our work easier and get the results quicker.

讓我們在一張圖中使用所有函數。現在，我們將為每個類別繪制一個散點圖，并使用所有功能。在此圖中，我們將添加一個按鈕，從中可以選擇所需的功能以使我們的工作更輕松并更快地獲得結果。

所有功能的匯總-一幅圖中的所有功能 (Aggregation of all functions — all functions in one-graph)

#Include the Librarylibrary(plotly)

#Store the graph in one variable to make it easier to manipulate.s <- schema()agg <- s$transforms$aggregate$attributes$aggregations$items$aggregation$func$valuesl = list()

for (i in 1:length(agg)) {     ll = list(method = "restyle",     args = list('transforms[0].aggregations[0].func', agg[i]),     label = agg[i])     l[[i]] = ll     }

p <- plot_ly(     type = 'scatter',     x = iris$Species,     y = iris$Sepal.Length / iris$Sepal.Width,     mode = 'markers',     marker = list(          size = 20,          color = 'orange',          opacity = 0.8          ),     transforms = list(          list(               type = 'aggregate',               groups = iris$Species,               aggregations = list(                    list(                         target = 'y', func = 'avg', enabled = T                    )               )            )     )) %>%layout(     title = '<b>Plotly Aggregations by Satyam Chauhan</b><br>use     dropdown to change aggregation<br><b>Sepal ratio of Length to     Width</b>',     xaxis = list(title = 'Species'),     yaxis = list(title = 'Sepal ratio: Length/Width'),     updatemenus = list(          list(               x = 0.2,               y = 1.2,               xref = 'paper',               yref = 'paper',               yanchor = 'top',               buttons = l          )     ))

#Display the graphs

這是什么意思？ (What does this mean?)

We make a list where all the function attributes of aggregation are stored. We use this function to experiment with all the functions of Aggregations in R.

我們列出存儲聚合的所有功能屬性的列表。我們使用此功能來試驗R中聚合的所有功能。

A few of the graphs with different examples are shown below.

下面顯示了一些帶有不同示例的圖形。

該代碼的作用是什么？ (What does this code do exactly?)

First, a list is created as mentioned earlier, in which all the functions are stored. After the list is made, the y-axis is set to the ratio of Sepal.Length to Sepal.Width and x-axis is set to Species.

首先，如前所述創建一個列表，其中存儲了所有功能。列出后，將y軸設置為Sepal.Length與Sepal.Width的比率，將x軸設置為Species。

After calculating the ratio, the function transform is called in which the func = ‘avg’ is mentioned for just the starting phase. When we run this code and select the function ‘mode’, we get Fig. 3 (above), which shows that the mode of setosa is the least among the three at around 1.4. Mode tells that the ratio 1.4 is repeated the most times or that value is most likely to be sampled. The different pattern we saw here is that the highest value most likely to be sampled is from the category veriscolor having a mode near to 2.2.

在計算出比率之后，將調用函數變換，其中僅在開始階段就提到了func ='avg'。當我們運行此代碼并選擇函數“ mode”時，我們得到圖3(上方)，該圖表明setosa的模式在這三個模式中最小，約為1.4。模式表明，比率1.4重復最多，或者最有可能被采樣。我們在這里看到的不同模式是，最有可能被采樣的最高值來自veriscolor類別，其模式接近2.2。

In Fig. 4 above, the change of ratio of Sepal Length to Sepal Width is plotted and we get very different results compared to the rest of the graphs. We observe the change of Setosa and Virginica to be the same and positive, while in the change of ratio by species, veriscolor is almost negative and is three times the change of the setosa and virginica.

在上面的圖4中，繪制了Sepal Length與Sepal Width之比的變化圖，與其余圖表相比，我們得到了截然不同的結果。我們觀察到Setosa和Virginica的變化相同且為正，而在物種比例變化中，veriscolor幾乎為負，是Setosa和virginica的三倍。

On the other hand, the right figure shows the rms values of each species. We can easily see that the species veriscolor and virginica have almost same value which is significantly greater than the rms value of setosa.

另一方面，右圖顯示了每種物質的均方根值。我們可以很容易地看到，veriscolor和virginica物種的值幾乎相同，大大高于setosa的rms值。

結論 (Conclusion)

Aggregation functions are one of the most powerful tools developers can ask for. They can provide you the patterns and results that you wouldn’t expect. To analyse the data visually, you have to play with the data, and to do that we need to manipulate and transform it. Aggregation functions do that for you, and they’re one of the most widely used functions in transform. This article is just a start. You can certainly explore more and apply more. That’s what explorers do.

聚合功能是開發人員可以要求的最強大的工具之一。他們可以為您提供意想不到的模式和結果。要以可視方式分析數據，您必須處理數據，并且為此，我們需要對數據進行操作和轉換。聚合函數可以為您做到這一點，它們是transform中使用最廣泛的函數之一。本文只是一個開始。您當然可以探索更多并應用更多。那就是探險家所做的。