A framework for mobile activity recognition

TL;DR: A hybrid method that integrates Latent Dirichlet Allocation with conventional classifiers for learning a generic activity model with minimum annotated data is proposed and a framework for low-level activity recognition with dynamically available sensors is proposed.

Abstract: Activity recognition is being applied in an increasing number of applications. They include health monitoring of the elderly, discovery of frequent behavioural patterns, monitoring of daily life activities (e.g. eating, tooth brushing, sleeping), and analysis of exercise activities (e.g. swimming, running). Current approaches for activity recognition usually use the process of data preprocessing, feature extraction, activity model learning and activity recognition. Most of the previous research pipeline these steps and create static models for processing activity data and recognizing activities. The static models have predefined data sources that are tightly coupled with the models and never change once the models are created. However, the static models are unable to deal with sensor failures and sensor replacements that are quite common in real scenarios. Moreover, additional information provided by newly available data sources from dynamically discovered new sensors may potentially refine the activity model if this information can discriminatively characterize a specific activity class. However, the static models cannot leverage this additional information for self-refinement due to the static assumption of data sources. The primary goal of our research is to design and develop frameworks for activity recognition with dynamically available data sources, and propose and develop algorithms for activity model adaptation with the additional information provided by those data sources. In this thesis, we first provide a critical literature review in the areas of contexts modelling, context management, sensor modelling and sensors in mobile devices, activity recognition, activity model retraining and adaptation, and sensor dynamics in activity recognition. We then present the research on our activity recognition framework that makes the following key contributions. First, we propose a hybrid method that integrates Latent Dirichlet Allocation with conventional classifiers for learning a generic activity model with minimum annotated data. The hybrid method is able to alleviate the problem of data sparsity and requires a little amount of labelled activity data. Furthermore, it can deal with different variants of activity patterns since it is created with activity data of multiple users. The generic activity modelling serves as the starting point of our activity model adaptation with dynamically available sensor data. However, it can also serve as an independent component for other applications such as activity personalization. Second, based on the generic model, we propose a framework for low-level activity (e.g. running, walking) recognition with dynamically available sensors. The components of the framework include a basic classifier, instance selection and smoothing. Firstly, we use AdaBoost as our basic classifier as it is flexible with feature dimensionality and it can automatically select the discriminative features during the learning process. Secondly, we propose to select the most informative instances for activity model adaptation in an unsupervised manner. The instances contain features of the new sensor data, and the information of new sensors are incorporated seamlessly through the adaptation process. Finally, we design smoothing methods by integrating the graphical models such as Hidden Markov Model and Conditional Random Field with the basic classifier AdaBoost. Finally, we propose a framework for high-level activity (e.g, making coffee) recognition with dynamically available contexts. We propose sensor and activity models to address sensor heterogeneity and populating contextual information. Knowledge-driven and data-driven methods are proposed for incorporating the new contexts. The knowledge-driven method specifies the parameters of the new contexts with external knowledge in an unsupervised manner, and the data-driven method learns the parameters of the new contexts with the users' data using the proposed learning-to-rank technique and temporal regularization. Extensive experiments and comprehensive comparisons demonstrate the effectiveness of the proposed frameworks.