drug1996 / deepnar Goto Github PK

Research Trend

• Compare the performance of different date of the art HAR method for very similar actions;
• Compare the performance of the method by using the different body part's data;
• Compare the performance of the method by using different sensor modalities individually and combination (data fusion)
• Transfer learning;
• Siamese network structure;
• Attention mechanism;
• Make modification on the structure of Deep-BiRNNs;
• Incremental learning to make the model personalized;
• Explain how RNN works in different layers (Layer visualization);
• OpenPose;

Mini-thesis

mini-thesis_zhong.pdf

Conferece

IJCAI (2016)
16_comparing_deep_and_classical_machine_learning_methods_for_human_activity_recognition_using_wrist_accelerometer.pdf

Author and affiliation

Author: Hristijan Gjoreski, Jani Bizjak, Martin Gjoreski, Matjaž Gams
Affiliation: Jožef Stefan Institute, Department of Intelligent Systems, Jožef Stefan International postgraduate School

Motivation

Har by only using wrist accelerometer is difficult. However, recently the wristband devices and smartwatches are becoming popular accompanied by recent trends in deep learning.

Database

Public database: Opportunity dataset
Self-made database (10 subjects)

Method

• Deep Learning: CNN
• Classical Machine Learning: J48 decision tree, Random Forest, Naive Bayes, SVM, and KNN
• leave-one-person-out cross-validation technique was used
  
  

Contribution summary (one-sentence)

They compared the recognition performance of DL-CNN and ML-RF method on two datasets, which showed the power of the DL to automatically extract relevant features and to achieve slightly better performance.

Useful information

• Numerous studies have shown that the performance of a HAR system strongly depends on the accelerometer placement;
• For daily activity database, the number of samples for each action class is imbalance;

Conferece

ACMMM (2015)
15_human_activity_recognition_using_wearable_sensors_by_deep_convolutional_neural_networks.pdf

Author and affiliation

Author: Wenchao Jiang, Zhaozheng Yin
Affiliation: Department of Computer Science, Missouri University of Science and Technology

Motivation

Achieve high recognition accuracy with low computational cost rather than exploring hand-crafted features from time-series signals.

Database

Public database: UCI, USC, SHO

Method

• Create a Signal Image, a 2D array to store permutated raw signals
• Apply 2D Discrete Fourier Transform (DFT) to the Signal Image to get Activity Image
• Build DCNN model
• Use SVM when the outputs of DCNN are not confident (called DCNN+)
  
  

Contribution summary (one-sentence)

Attacks the problem of accurate and efficient HAR based on wearable sensors by using DCNN.

Insight

Create a pre-processing method to make multi-channel time series adapt to CNN model;
Combine two models (CNN and SVM) to get better performance.

Journals

International Journal of Machine Learning and Computing (Dec 2018)
18_comparative_study_of_machine_learning_and_deep_learning_architecture_for_har_using_accelerometer_data.pdf

Author

Author: Sarbagya Ratna Shakya, Chaoyang Zhang, and Zhaoxian Zhou

Motivation

har has been a popular field of research in recent times, and many approaches have been implemented

Database

Public database: WISDM, Shoaib SA

Method

• ML: RF, DT, KNN (without hand-crafted feature extraction)
• DL: CNN, RNN

Contribution

The experiment results from this study provide a comparative performance analysis based on the accuracy and performance of different existing ML algorithms and DL algorithms.

Insight

Just compare and analyze the performance of different existing algorithms of ml and dl for har

Useful information

• Gyroscopes can have higher accuracy than accelerometer data;
• Putting sensors on the hip provides better accuracy than at other locations of the body;
• Average loss for one epoch rather than a batch;
• DL algorithms get better performance in the balanced dataset;
• DL algorithms can have higher accuracy with the data from multiple accelerometer sensors than with a single one;
• Early stop;

Journals

Sensors (2017)
17_deep_recurrent_neural_network_for_har.pdf

Author and affiliation

Author: Abdulmajid Murad and Jae-Young Pyun
Affiliation: Department of Information Communication Engineering, Chosun University, 375 Susuk-dong, Dong-gu, Gwangju 501-759, Korea

Motivation

Typical models (typical machine learning and CNN) are unadaptable to a wide range of activity-recognition configurations and require fixed-length input windows.

Database

Public database: UCI-HAD, USC-HAD, Opportunity, Daphnet FOG, Skoda

Method

• Three proposed structures: Unidirectional DRNNs, Bidirectional DRNNs, Cascaded unidirectional and bidirectional DRNNs
• Dropout between the layesrs
• Used late-fusion technique in which the classification decision are combined for the overall prediction of a window;

Unidirectional LSTM-based DRNNs

Bidirectional LSTM-based DRNNs

Cascaded unidirectional and bidirectional LSTM-based DRNNs

Contribution

They presented three LSTM-based DRNN architectures for HAR, which outperform other state-of-the-art methods by testing on four benchmark datasets.

Results of the proposed model for the Opportunity dataset

Insight

• When training the model, there is no valid set. The model and the result cannot be trusted, which may be overfitted to the test set;
• Borrow Deep-RNNs from other domain and test on HAR;

Useful information

The optimal window length of a dataset depends on the sampling rate and the type of activities performed;

Journals

Mathematical Problems in Engineering (2018)
18_deep_residual_bi_lstm_for_har_using_wearable_sensors.pdf

Author and affiliation

Author: Yu Zhao , Rennong Yang , Guillaume Chevalier , Ximeng Xu , and Zhenxing Zhang
Affiliation: Aeronautics and Astronautics Engineering College, Air Force Engineering University, Xi'an 710038, China; Laval University, 2325 Rue de l'Universite, Quebec G1V 0A6, Canada

Motivation

Aim to enhance the recognition rate for har

Database

Public database: UCI, Opportunity

Method

• residual network of rnn
  

Contribution

Proposed a framework called Res-Bidir-LSTM, which improved the accuracy on UCI and Opportunity compared with the previous work they choosed.

Results of the proposed model for the Opportunity dataset

Insight

• Borrow the idea from Microsoft's resnet;
• The author mentioned DeepConvLSTM, but didn't compare the proposed model with it, since the proposed model's performance is not better than DeepConvLSTM;
• The model may overfit the test dataset since there was no valid dataset;

Useful information

• Use mean and variance normalization for preprocessing;
• L2 norm for weight decay and dropout are used to prevent overfitting, and gradient clipping and batch normalization are used to prevent gradient vanishing or explosion as well as overshooting the learning updates;
• Grid search for hyper-parameters;
• Deal with the missing data;

Conferece

IJCAI (2015)
15_deep_convolutional_neural_networks_on_multichannel_time_series_for_human_activity_recognition.pdf

Author and affiliation

Author: Jian Bo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, Shonali Krishnaswamy
Affiliation: Data Analytics Department, Institute for Infocomm Research, A*STAR, Singapore 138632

Motivation

Most existing work relies on heuristic hand-crafted feature design which cannot find distinguishing features to accurately classify different activities.

Database

Public database: Opportunity Activity Recognition Challenge, Hand Gesture

Method

• Use sliding windows to get instances with the window size of sampling frequency
• Build a CNN model
• Employ a low-pass filter to post-process predicted labels
  

Contribution summary (one-sentence)

Proposed a new method to automate feature extraction for the human activity recognition task at that time.

Useful information

The reasons that signal acquired by on-body sensors are arguably favorable over the signal acquired by video cameras.

Insight

At that time, deep learning models have not been fully exploited in the field of HAR.

Journals

Sensors (2016)
16_deep_convolutional_and_lstm_rnn_for_multimodal_wearable_activity_recognition.pdf

Author and affiliation

Author: Francisco Javier Ordóñez and Daniel Roggen
Affiliation: Wearable Technologies, Sensor Technology Research Centre, University of Sussex, Brighton BN1 9RH, UK

Motivation

Enhancing recognition accuracy and decreasing reliance on engineered features to address increasingly complex recognition problems.

Database

Public database: Opportunity, Skoda

Method

• Combine CNN and LSTM (DeepConvLSTM) to build the network (borrow from speech recognition domain)
• Compare the performance with CNN
• Test the performance of using different sensor modalities
  

Contribution

The authors demonstrated the advantages of a deep architecture based on the combination of CNN and RNN to perform activity recognition from wearable sensors, compared with CNN.

Insight

Borrow a new structure of neural network from other domain to apply it on the unexplored domain.

Useful information

• naturalistic human activities are often highly class unbalanced;
• recurrent model is capable of obtaining very good performance with relatively small datasets;
• how sliding windows works and how to evaluate and compare;
  
  

test

Journals

IEEE Internet of Things Journal (2016)
16_multicolumn_bi_lstm_for_mobile_devices-based_human_activity_recognition.pdf

Author

Dapeng Tao, Yonggang Wen, Senior Member, IEEE, and Richang Hong

Motivation

Improve the performance of mobile device-based human activity recognition system (MARSs).

Database

Self-made database (100 subjects)
7 classes: jumping, running, walking, step walking, walking quickly, downstairs, upstairs

Method

• Present a new two-directional feature (data preprocessing)
• Combine several BLSTM classifiers to form an MBLSTM
• Incremental learning structure
  

Contribution

They present a feature descriptor, the two-directional feature, combined with multi-column BLSTM to improve activity recognition.

Insight

• Averaging the outputs of several BLSTMs, each has different input with others created by data preprocessing
• Incorporate incremental learning into the system