Date of Award

5-12-2024

Degree Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Electrical Engineering and Computer Science

Advisor(s)

Vir Phoha

Second Advisor

Asif Salekin

Subject Categories

Computer Sciences | Physical Sciences and Mathematics

Abstract

The rapid advancements in deep learning and smart hardware have accelerated the development of various automatic human-centric sensing applications. However, the intrusive nature of these sensing applications and the heterogeneity of sensory data pose challenges in real-world deployment. While performance is crucial, ensuring the security and robustness of these applications is equally imperative for their reliable operation. To tackle the challenges associated with security and robustness in human-centric sensing, this dissertation outlines two specific objectives: (1) mitigating false data injection attacks (FDIA) on sensing applications, and (2) establishing a generalized personalization framework for human sensing models. FDIA operates by injecting signals from individuals with specific traits into the victim’s sensory data stream, such a mixture of forged and valid signals can successfully deceive the continuous authentication system to accept it as an authentic signal. Simultaneously, introducing a targeted trait in the signal misleads human-centric applications to generate specific targeted inferences, which may cause adverse effects. In the first objective, we evaluate the FDIA’s deception efficacy on sensor-based authentication and human-centric sensing applications simultaneously using two modalities - accelerometer, and blood volume pulse signals. We identify variations of the FDIA such as different forged signal ratios, smoothed and non-smoothed attack samples. Notably, we present a novel attack detection framework named Siamese-MIL that leverages the Siamese neural networks’ generalizable discriminative capability and multiple instance learning paradigms through a unique sensor data representation. Our exhaustive evaluation demonstrates Siamese-MIL’s real-time execution capability and high efficacy in different attack variations, sensors, and applications. The distinct patterns in human-centric sensing, influenced by various factors or contexts, pose challenges to generic model performance due to natural distribution shifts. Researchers address this challenge through domain adaptation approaches, with personalization being a widely used method that tailors generic models to individual users, enhancing performance. However, sensory data exhibits intra-user heterogeneity across contexts, an aspect overlooked by existing personalization studies, leading to a lack of individual-level generalizability in contexts not present during personalization. The second objective investigates this issue and introduces a framework called GenPersonalization to fill this research gap effectively. GenPersonalization aims to personalize a generalized generic model to the target user’s traits and maintain generalizability to other contexts while using limited data from one (or limited) context. GenPersonalization mitigates the generalizability loss challenge during personalization by introducing a novel approach using the original generic model as an anchor to retain knowledge about other contexts. Extensive evaluations across three human-centric sensing applications and multiple sensor modalities in both lab-controlled environments and real-world settings show the natural intra-user distribution shift compromises state-of-the-art personalized models' generalizability, while GenPersonalization effectively balances performance and generalizability across contexts.

Access

Open Access

Download

Share

COinS