I have found the DeepSense framework as one of the promising deep learning architectures for processing Time-Series sensing data. In this brief and intuitive overview, I’ll present the main ideas of the original paper titled “Deep Sense: A Unified DL Framework for Time-Series Mobile Sensing Data Processing” by Yao et-al. (Can be found at www2017).

我發現DeepSense框架是用于處理時間序列感應數據的有前途的深度學習架構之一。 在這個簡短而直觀的概述中，我將介紹Yao等人題為“深度感知：用于時間序列移動傳感數據處理的統一DL框架”的原始論文的主要思想。 (可以在www2017上找到)。

主要問題 (The Main Problem)

DeepSense addresses the problem of mobile sensing using time-series data from various sensors (accelerometers, gyroscopes, and more). Due to the quality of the sensor, processing any type of sensor involves noisy measurements. It is difficult to find a distribution that describes noise in real-life, so in the end, our measurements are corrupted by ”unknown” noise (nonlinear, correlated over time, etc.). In addition, for tracking applications, a physical system model should be defined. This model is also affected by “noise” in terms of inaccuracy (as we cannot predict/ define the model without error). As we deal with real-time measurement (and physical model), the behavior can be described as “time-series measurements of physical quantities..”.

DeepSense使用來自各種傳感器(加速度計，陀螺儀等)的時間序列數據解決了移動感應的問題。 由于傳感器的質量，處理任何類型的傳感器都會涉及噪聲測量。 很難找到描述現實生活中的噪聲的分布，因此最后，我們的測量結果被“未知”噪聲(非線性，隨時間相關等)破壞了。 此外，對于跟蹤應用程序，應定義一個物理系統模型。 該模型在準確性方面也受到“噪聲”的影響(因為我們無法無誤地預測/定義模型)。 當我們處理實時測量(和物理模型)時，該行為可以描述為“物理量的時間序列測量”。

(核心)想法 (The (core) Idea)

The author proposes a unified DL framework that addresses the challenges described in mobile sensing: integration of Convolution Neural Networks (CNN) and Recurrent Neural Network (RNN). CNN is responsible for the computation of the sensing quantities within the time interval where it extracts the local features for all sensors and combines them into global features. RNN is responsible for computation of the sensing quantities across time intervals, where it extracts temporal dependencies. The DeepSense solves both classification and regression of mobile computing tasks in some unified manner.

作者提出了一個統一的DL框架，以解決移動傳感中描述的挑戰：卷積神經網絡(CNN)和遞歸神經網絡(RNN)的集成。 CNN負責計算時間間隔內的感應量， 在該時間間隔內 ，CNN提取所有傳感器的局部特征并將其組合為全局特征。 RNN負責跨時間間隔傳感量的計算，在那里它提取時間相關。 DeepSense以某種統一的方式解決了移動計算任務的分類和回歸問題。

架構 (The Architecture)

The architecture consists of 3 parts: a convolutional layer, recurrent layer, and output layer. Since DeepSense can be used for classification and regression tasks, the output layer should be set according to the specific task.

該體系結構由三部分組成：卷積層，循環層和輸出層。 由于DeepSense可用于分類和回歸任務，因此應根據特定任務設置輸出層。

Convolutional layer: an individual convolutional subnet can be represented by 3 layers (Individual Convolutional Layers 1,2,3). By applying a 2d filter, the net can learn interaction among sensor measurements and local patterns. Next, by Flatten & Concatenation layer, the matrices are flattened into a vector which is the input to the next 3 layers: The Merge Convolutional Layers. A 2d filter is also applied here to learn the interaction between all K input sensors. For each convolutional layer, DeepSense learns 64 filters by using ReLU. Apply batch normalization at each layer to reduce the internal covariate shift. A Flatten & Concatenation layer completes this phase of the convolutional layer.

卷積層：單個卷積子網可以由3層表示( 單個卷積層 1,2,3)。 通過應用二維過濾器，網絡可以了解傳感器測量值和局部模式之間的相互作用。 接下來，通過Flatten＆Concatenation層，將矩陣展平為一個向量，該向量是接下來3層的輸入： 合并卷積層。 這里還應用了2d濾波器，以了解所有K個輸入傳感器之間的相互作用。 對于每個卷積層，DeepSense通過使用ReLU學習64個過濾器。 在每一層應用批量歸一化以減少內部協變量偏移。 Flatten＆Concatenation層完成了卷積層的此階段。

Recurrent layer: the power of RNN is in the ability to approximate function and understand the important features for time-series. In this architecture, the RNN extension model that used is a stacked Gated Recurrent Unit (GRU). 2 layers of RNN are implemented with dropout to the connection between these Recurrent Layers (1,2), as also recurrent batch normalization in order to reduce the internal covariance shift among data series.

循環層： RNN的功能在于能夠逼近函數并了解時間序列的重要特征。 在此體系結構中，使用的RNN擴展模型是堆疊的門控循環單元(GRU)。 RNN的2層實現為與這些遞歸層(1,2)之間的連接斷開，也為遞歸批歸一化，以減少數據序列之間的內部協方差漂移。

Output Layer: Up to this phase, the net has a series of vectors for each time step. Now, we should handle carefully upon the tasks: regression or classification. For regression, a dictionary should be learned. For classification, the composition of the output layer should be done by averaging the features over time. then, feeding the final features into a softmax layer to generate the prediction.

輸出層：到此階段，網絡每個時間步都有一系列矢量。 現在，我們應該認真處理以下任務：回歸或分類。 為了回歸，應該學習字典。 對于分類，輸出層的組成應通過對特征隨時間進行平均來完成。 然后，將最終特征輸入softmax層以生成預測。

摘要和我的觀點 (Summary and My Point of View)

DeepSense seems to be very promising for many time-series tasks, Aside the Temporal Convolutional Networks, WaveNet, and others. When I first read the paper, I thought about the Extended Kalman Filter as a nice classical compression — as it deals with the same type of problem. This deep learning architecture might deal with model uncertainty, noisy measurement, and more.

除了時間卷積網絡，WaveNet等，DeepSense對于許多時間序列任務似乎都非常有前途。 當我第一次閱讀本文時，我認為擴展卡爾曼濾波器是一種很好的經典壓縮方法，因為它可以處理相同類型的問題。 這種深度學習架構可能會處理模型不確定性，噪聲測量等問題。

Further reading

進一步閱讀

Yao, Shuochao, et al. “Deepsense: A unified deep learning framework for time-series mobile sensing data processing.” Proceedings of the 26th International Conference on World Wide Web. 2017.

姚碩超，等。 “ Deepsense：用于時間序列移動感測數據處理的統一深度學習框架。” 第26屆國際萬維網會議論文集 。 2017。