Link to website: AA06

The objective was to apply computer vision techniques based on Artificial Intelligence for human pose detection that gives real-time feedback. The system assesses a yoga pose being performed and gives real time feedback in the form of error indicators along with a score to measure the accuracy of the pose being performed.

Recent advancements in Deep Learning has added a huge boost to the already rapidly developing field of computer vision. Pose estimation is a subset of computer vision that refers to computer vision techniques that detect human figures in images and video, so that one could determine, for example, where someone’s ankle shows up in an image. The motivation behind this project was to detect movement and posture errors in performing yoga poses. The idea was to build an effective yoga programme for each user to help them improve their posture and health.

Building a human pose detection system with the PoseNet model entailed dividing the process into 3 steps: Data Collection, Model Training and Model Testing.

Data Collection: Using PoseNet, when a pose would be detected, the 14-(x,y) inputs and target data would be fed into the neural network defined object. A setTimeout() function was used to record the poses for 5 seconds.

Model Training: The existing data would be loaded into the JavaScript environment. Once the downloaded json file was attached to the model training file, the neural network was then trained across 50 epochs.

The performance of the model is quite promising. For poses such as ‘Mountain’ and ‘Warrior 2’, the results, when performed correctly, classifies the current pose.

Upon evaluation of the model, the model is able to classify the poses with an accuracy of 82.9%. Incorrect poses can be detected immediately, and outputs real-time feedback to the user with error indicators.

The performance of the model drops when the detection of keypoints fails. The scenarios include dark background or environment, as well as multiple users being in the frame of recording. The pose estimation uses a single-pose estimation model to perform data points collection, before being fed to the model classifier.

Attempt of performing ‘Tree’ pose with some computed angles falling below a threshold of 10° and 20° error:

Attempt of performing ‘Goddess’ pose with some computed angles falling below margin of 20° error:

Attempt of performing ‘Goddess’ pose with some computed angles falling below a threshold of 10° and 20° error:

Attempt of performing ‘Warrior 2’ pose with some computed angles falling below a threshold of 10° and 20° error: