Determining the similarity between abstract scenes is a difficult problem in cognitive computing. Humans can look at two images or videos and immediately tell if they are similar, but this intuition is difficult to encode.

We present an application of 2D human pose estimation for accessing similarity between motions in the space of weightlifting. Given input video of a person performing an exercise, our system will access their form. That is, it will give a similarity score between the user's motion and that motion performed "ideally".

Clone the public repository

git clone https://github.com/RFinkelberg/Stance.git

Use the package manager pip to install the dependencies

pip install -r requirements.txt

Download the neural net

python get_models.py

cd stance/
python main.py -f example/squat.mp4

usage: main.py [-h] [-f VIDEO_PATH] [-v] [-p]

optional arguments:
  -h, --help            show this help message and exit
  -f VIDEO_PATH, --video_path VIDEO_PATH
                        filepath to video containing the user performing a
                        motion
  -v, --verbose         Verbosity level. 1 (v) displays info, 2 (vv) displays
                        debug logs
  -p, --use_pickle      uses the pickle file corresponding to the video file
                        given
  -s, --save_pickle     makes a pickle file corresponding to the video file
                        given

There are three pickle files each containing precomputed skeletons for each frame of the videos. To use, simply attach a "-p" to the end of the command with the relevant video file.

• example/squat.mp4
• example/squatbad.mp4
• example/squatpoop.mp4

python main.py -f example/squatbad.mp4 -p

If you would like to save the output of the neural net to pickle file so the system spends less time computing the user skeletons of the video in future runs, simply attach a "-s" to the end of the command.

python main.py -f example/squat.mp4 -s

• Jorge Betancourt
• Roy Finkelberg
• Aadarsh Padiyath
• Emilee Sisson

This project is licensed under the MIT License - see the LICENSE.md file for details

For more information on how we developed this project, check out our stance website.