Date of Award
December 2018
Degree Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Electrical Engineering and Computer Science
Advisor(s)
Senem Velipasalar
Second Advisor
Leanne M. Hirshfield
Keywords
Brain activity, Deep Learning, fNIRS, Human Computer Interaction, Machine Learning, Multi channel sensor
Subject Categories
Engineering
Abstract
Identification of user state is of interest in a wide range of disciplines that fall under the umbrella of human machine interaction. Functional Near Infra-Red Spectroscopy (fNIRS) device is a relatively new device that enables inference of brain activity through non-invasively pulsing infra-red light into the brain. The fNIRS device is particularly useful as it has a better spatial resolution than the Electroencephalograph (EEG) device that is most commonly used in Human Computer Interaction studies under ecologically valid settings. But this key advantage of fNIRS device is underutilized in current literature in the fNIRS domain.
We propose machine learning methods that capture this spatial nature of the human brain activity using a novel preprocessing method that uses `Region of Interest' based feature extraction. Experiments show that this method outperforms the F1 score achieved previously in classifying `low' vs `high' valence state of a user.
We further our analysis by applying a Convolutional Neural Network (CNN) to the fNIRS data, thus preserving the spatial structure of the data and treating the data similar to a series of images to be classified. Going further, we use a combination of CNN and Long Short-Term Memory (LSTM) to capture the spatial and temporal behavior of the fNIRS data, thus treating it similar to a video classification problem. We show that this method improves upon the accuracy previously obtained by valence classification methods using EEG or fNIRS devices. Finally, we apply the above model to a problem in classifying combined task-load and performance in an across-subject, across-task scenario of a Human Machine Teaming environment in order to achieve optimal productivity of the system.
Access
Open Access
Recommended Citation
Bandara, Danushka Sandaruwan, "Machine Learning Methods for functional Near Infrared Spectroscopy" (2018). Dissertations - ALL. 953.
https://surface.syr.edu/etd/953
