The implementation of STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising in ISMAR 2021.

We propose a method for denoising hand motion data in mixed reality using a spatial-temporal graph auto-encoder. Our approach models the articulated hand structure with a partition strategy and extracts structural constraints using graph convolution and self-attention. We combine graph and temporal convolutions in an hourglass residual auto-encoder to preserve structural constraints while denoising hand motion data. Our method outperforms state-of-the-art approaches in both quantitative and qualitative experiments.

Paper Address | Download Paper | Supplementary Video | Multi-STGAE

• Denoising results

More demos can be seen in our supplementary video.

• LaTeX tool

sudo apt-get install texlive-full

• FFmpeg

sudo apt-get install ffmpeg

• pydot & graphviz

sudo pip3 install pydot
sudo pip3 install graphviz

The core code of the client and server is available, and detailed development tips can be found at the link

@inproceedings{zhou2021stgae,
  title={STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising},
  author={Zhou, Kanglei and Cheng, Zhiyuan and Shum, Hubert PH and Li, Frederick WB and Liang, Xiaohui},
  booktitle={2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
  pages={41--49},
  year={2021},
  organization={IEEE}
}

Feel free to contact me via zhoukanglei[at]qq.com.