Feature engineering on high-dimensional data for better performance

Approximately 50% of people with Spinal Cord Injury (SCI) have Central Neuropathic Pain (CNP). Pain in response to non-painful stimuli, episodic (electric shock), “pins and needles”, numbness. There is currently no treatment, only prevention Preventative medications have strong side-effects Predicting whether a patient is likely to develop pain is useful for selective treatment Manual assessment is time-consuming, error-prone, and somewhat subjective There is some evidence that brain Electroencephalogram (EEG) data has characteristic markers. We have a (small) dataset with EEG from SCI patients, of which some later developed CNP. The data is extremely high-dimensional, so it is very hard for a classifier to tell them apart. To overcome this curse of dimensionality problem we are doing some feature engineering and feature selection techniques and extracting a small subset of useful features that will result in better performance of our machine learning model.

This is a group project done as a part of my masters program