Augmenting Deep Learning Adaptation for Wearable Sensor Data through Combined Temporal-Frequency Image Encoding

Question

1. What challenges need to be addressed for wearable sensor data in deep learning models?

2. What are the limitations of existing activity recognition approaches?

3. How does the modified recurrence plot method incorporate frequency domain information?

4. What algorithms were implemented for benchmarking activity recognition?

Accepted Answer

To extend the applicability of deep learning models to wearable sensor data, several challenges need to be addressed. These include preprocessing, artifact removal, noise reduction, and careful modification of advanced deep learning techniques. Additionally, substantial efforts are required for data collection to facilitate the pretraining process. Existing pretrained computer vision-based models, such as RestNet and AlexNET, are rendered ineffective in the context of wearable sensors. Researchers have proposed converting wearable sensor data into image representations, predominantly using Recurrent Plots in the temporal domain. In this paper, a novel modified-recurrent plot-based image representation for wearable sensor data that incorporates both temporal and frequency domain information is introduced.

Accepted Answer

Existing activity recognition approaches have limitations in terms of extensive preprocessing, scalability, and reliance on feature extraction quality and domain-specific knowledge. They often focus on time-domain information and do not incorporate frequency domain information, resulting in stagnant accuracy rates of 93%.

Accepted Answer

The modified recurrence plot method incorporates frequency domain information by calculating the Fourier transform of two temporal phases within their time-window, resulting in complex-valued frequency spectra. The phase of each frequency component is computed, and the angle between a base vector and the phase difference vector is used to distinguish different gradient directions. This method helps identify the quadrant in which the state difference vector falls, distinguishing uphill and downhill tendencies. By using this approach, the modified recurrence plot method improves the encoding of accelerometer signals as images, providing more detailed information about the dynamic system being studied.

Accepted Answer

The implemented algorithms for benchmarking activity recognition include Random Forest (RF), Support Vector Machines (SVM), Convolutional Neural Network (CNN), Dynamic Time Warping (DTW) + 1 Dimensional CNN, DTW + Clustering, Long Short Term Memory (LSTM) + Fully Connected Neural (FCN) network, Temporal RP + ResNet (TRP+ResNet), modified Temporal RP + ResNet (MTRP+ResNet), Frequency domain RP plot with ResNet architecture (FRP+ResNet), and Mixup Augmentation plot of temporal and frequency (Our Method). These algorithms were used to assess the individual contributions of the proposed method and compare their performance against baseline algorithms.

Accepted Answer

Accuracy was used as the evaluation metric, calculated as the ratio of true positive and true negative predictions to the total number of predictions. The standard error was also used as a measure of the distribution of errors. The accuracy was determined by following a user-mixed approach, which involved combining all episodes, splitting the data into a 70:30 ratio for training and testing datasets, and performing training on the training data with 20% random data selected as validation data during each iteration of neural network training. Finally, the performance was evaluated on the test data. Table I and Table II present the accuracy comparisons of our proposed method with different baseline algorithms.

Accepted Answer

The limitations of the proposed model include its validation only on wearable accelerometer sensor signals, the need for further validation on various time-series data such as EEG, EDA, PPG, Gyroscope, and Magnetometer, and the evaluation solely on classification problems. Additionally, it is necessary to validate the model with other scalable machine learning techniques like active learning, opportunistic learning, transfer learning, and reinforcement learning. The long-term goal is to develop Automatic Scalable Machine Learning (AutoScaleML) that can represent any time-series signal using appropriate image representations that combine both temporal and frequency domain information.

Augmenting Deep Learning Adaptation for Wearable Sensor Data through Combined Temporal-Frequency Image Encoding

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What challenges need to be addressed for wearable sensor data in deep learning models?

2. What are the limitations of existing activity recognition approaches?

3. How does the modified recurrence plot method incorporate frequency domain information?

4. What algorithms were implemented for benchmarking activity recognition?

5. How did accuracy measure performance?

6. What are the limitations of the proposed model?

Related Papers (5)

Image Preprocessing Techniques in Skin Diseases Prediction using Deep Learning: A Review

Comparison of Data Preprocessing Approaches for Applying Deep Learning to Human Activity Recognition in the Context of Industry 4.0.

FLIRT: A feature generation toolkit for wearable data.

Ship Detection from X-Band SAR Images Using M2Det Deep Learning Model

Deep Learning in Automating Breast Cancer Diagnosis from Microscopy Images