Check out the documentation of fimi for full information about the whole system.

fimi is an application which tries to resemble a coach while you doing workout by analyzing your posture and giving you feedback about what you should watch more often. To run it, you need a webcam, a big screen and your smartphone (and some room space). You control the big screen with your smartphone by simply scanning the displayed QR code (being in the same wifi net is necessary).

fimi generates a 3d pose estimation based on the webcam image and checks every frame your posture based on a fixed set of rules per exercise. In the end, you receive a small summary of your results on your smartphone.

The application also compares your posture with your previous trainings while still being designed with privacy in mind. All your information is stored only temporarily while running the application on your PC or Mac, whereas the training data is stored on your smartphone directly. This also means that you could theoretically log in at your friend's house and have your profile with you.

This is the pose estimation server of the fimi fitness mirror application.

• XNECT C++ Library by Max Planck Institute, revised and adapted
• Caffe for Windows
  • NVIDIA CUDA v9.2
  • Python 2.7
  • OpenCV 3.1
  • GFlags
  • HDF5
  • LevelDB
  • LMDB
  • Protobuf
  • Snappy
  • GCC
    • GFortran
    • LibQuadMath
  • ZLib
  • OpenBLAS
• WebSockeetPP

1. Python 2.7 (x64 version)
2. Visual Studio 2015 with Visual C++, Python and Git Plugin
3. NVIDIA CUDA 9.2
4. CMake
5. numpy x64 via pip install numpy

This is based on https://tkcheng.wordpress.com/2019/04/11/caffe-on-windows-rtx-2080ti-cuda9-2-vs2015/

1. After you have installed all dependencies above, clone Caffe (from XNECT branch: https://github.com/xu-chris/caffe/tree/net/xnect).

git clone https://github.com/xu-chris/caffe.git
cd caffe
git checkout net/xnect

2. Call

.\scripts\build_win.cmd

so Caffe downloads automatically the dependencies it needs.

Building Caffe will fail since we want to use a customized version of boost that supports our CUDA version and architecture.

For this, open ~\.caffe\dependencies\libraries_v140_x64_py35_1.1.0\libraries\include\boost-1_61\boost\config\compiler\nvcc.hpp and comment out the last three lines (commenting is done by adding // to it).

It looks then like this:

//#if !defined(__CUDACC_VER__) || (__CUDACC_VER__ < 70500)
//#   define BOOST_NO_CXX11_VARIADIC_TEMPLATES
//#endif

1. Add dependencies variables to PATH. On Windows, search via Cortana search bar for System Environment Variables and click on Environment Variables button in the lower right corner. Open Path variable and add the following below (change <your-username> to the user folder name of your current user):

C:\Users\<your-username>\.caffe\dependencies\libraries_v140_x64_py27_1.1.0\libraries\lib\cmake\glog
C:\Users\<your-username>\.caffe\dependencies\libraries_v140_x64_py27_1.1.0\libraries\cmake
C:\Users\<your-username>\.caffe\dependencies\libraries_v140_x64_py27_1.1.0\libraries\lib
C:\Users\<your-username>\.caffe\dependencies\libraries_v140_x64_py27_1.1.0\libraries\x64\vc14
C:\Users\<your-username>\.caffe\dependencies\libraries_v140_x64_py27_1.1.0\libraries\include
C:\Users\<your-username>\.caffe\dependencies\libraries_v140_x64_py27_1.1.0\libraries
C:\Users\<your-username>\.caffe\dependencies\libraries_v140_x64_py27_1.1.0\libraries\x64\vc14\bin

2. Clone full git repository:

git clone --recurse-submodules -j8 https://github.com/xu-chris/fimi-Server.git fimi-Server

3. Build it by calling (line by line)

cd fimi-Server
mkdir build
cd build
cmake -G "Visual Studio 14 Win64" -C ~\.caffe\dependencies\libraries_v140_x64_py27_1.1.0\libraries\caffe-builder-config.cmake ..

Open the generated pose_estimator.exe executive in the \bin\Release folder.

Check out the parameters in \data\FullBodyTracker\XNECT and adjust them to your needs.

The server has three different modes which can be switched by changing the parameter mode in \src\main.cpp to one of the following options:

• Mode::LIVE: Capture the current webcam image and process the pose estimation on it
• Mode::VIDEOINPUT: Fetch the video file stored in the parameter videoFilePath and process this. It also creates an output video and saves it in the path of the executable with the name YYYY-MM-DD-HH-MM-SS-pose_estimation_recording.avi.
• Mode::SIMULATION_RECORDING: This is useful when you want to use mock data and the fimi Mock Server for further development of the fimi Client. It fetches the video file stored in the parameter videoFilePath and processes this. It creates a test.mock file which contains per line: the delay in milliseconds as first value and the vector points as following.

You can set furthermore the following settings:

• PORT_NUMBER: The port which will be opened for the WebSocket Server to send the pose estimation to it's listeners.
• REPEAT_VIDEO: Set this to 1 if you want to keep the video repeating while playing in Mode::VIDEOINPUT mode. 0 otherwise.
• SHOW_WINDOW: Set this to 0 to hide the window and gain some of the processing speed and power. 1 otherwise. Default is 1.

• Caffe does not run with Python version >3.5: Please note that caffe only supports Python 3.5 for now. If you try it first with any higher version. delete the build folder and restart the process.
• Missing ZLIB_LIBRARY: Download ZLIB binary from the official homepage and install it. Add C:\Program Files (x86)\GnuWin32 to your PATH.
• Missing dependencies on executable start: In some cases, you need to add and build further dependencies, like zlib. If you need and do so, don't forget to add the path to the package into path of your system environment variables.