mad-lab-fau / bash-model Goto Github PK

We developed a method animating a statistical 3D human model for biomechanical analysis to increase accessibility for non-experts, like patients, athletes, or designers.

License: Apache License 2.0

C++ 97.92% GLSL 1.92% Batchfile 0.16%

statistical-model 3d-models scape opensim human-pose-estimation visualization biomechanics animated

bash-model's Introduction

BASH - Biomechanical Animated Skinned Human

Schleicher, R., Nitschke, M., Martschinke, J., Stamminger, M., Eskofier, B., Klucken, J., Koelewijn, A. (2021). BASH: Biomechanical Animated Skinned Human for Visualization of Kinematics and Muscle Activity. 16th International Conference on Computer Graphics Theory and Applications (GRAPP), 2021.

https://www.scitepress.org/Papers/2021/102106/102106.pdf

BASH Model

Converting a OpenSim [1] format file (.osim + .mot) to the SCAPE [2] framework. Visualization tool to inspect the animated model in 3D.

Processing Pipeline

Input Model: OpenSim

Parser
Model (.osim)
Scale factors (.xml)
Motion (.mot)
Muscle Activation (.sto)

Baseline model Design for a new Musculoskeltal Model (in Blender)

modeling
import SCAPE mesh
rig and skin skeleton (same hierarchy as musucloskeletal model)
place markers (same set as musculoskeletal model)
export model (.dae reorders vertices...) => mesh, markers & weights files

Scaling

performed automatically, applied correctly to the hierachy, applied in bone space
use .xml file or my estimation (defined in settings.h)
scaled vs generic in ./data/cache/mesh/

Initial Pose Matching

computed automatically using OpenSim's IK solver
cached in ./data/cache/mapping for debugging the resulting .mot file

Pose Transformation

calculated beforehand (needed the mesh for projection to SCAPE)
uses pose mapping projection and kinematic transformations, applied in world space
cached in ./data/cache/mesh/

Projection into SCAPE space

projection to scape space (only relative rotations)
rigid alignment to adjust translation
cached in ./data/cache/mesh/

Visualization of Muscle Activation

computed at run-time
color coding in Fragment Shader

Settings

settings.h for keyshortcuts, constants and other configurations

Project structure and dependencies

SCAPE: The main Windows-Application that handles the model conversion and visualization
External dependencies (minimum required version):
SFML (>= 2.5.1)
glew (>= 2.1.0)
glm (>= 0.9.9.5)
Assimp (>= 3.0.0)
OpenSim and SimbodyTK (>= 4.0)
libRender: A custom framework used for creating a window and render a 3D-application in it
External dependencies (minimum required version):
SFML (>= 2.5.1)
glew (>= 2.1.0)
glm (>= 0.9.9.5)
libSCAPE: The SCAPE framework to load the SCAPE binary data and create a mesh in pose and shape
External dependencies (minimum required version):
SuitSparse package: suitsparse + amd + umfpack (>= 5.1.2)
GSL (>= 2.4)

SCAPE Framework

Implementation in ´SCAPE.h´
Model parameters
Pose: Rotation vector for each part ('numParts = 16') in three-dimensional twist subvectors (the axis is determined by the vector's direction and the angle is determined by the vector's magnitude.
Shape: Scalar PCA coefficients ('numVecs = 20') to modify body proportions like height, size and gender etc.

Building platform x64

OpenSim can only be built in 64bit. So we have to use the x64 Platform in order to use their API.
Include and link all dependencies in x64.
Build the SCAPE framework in x64.
Define the flag '#define SAVE_MATRIX 0' once to write new binaries in the correct format (64bit wording).
The folder 'data\default_scape_data' should contain the binary files: 'matrixDGrad.bin', 'SCAPE_DGrad_numeric.bin', 'SCAPE_DGrad_symbolic.bin', 'SCAPE_pose.bin'.

Example result

OpenSim's visualization compared to our visualization (data set: straight running [3]):

References

[1] Seth, A., Hicks, J. L., Uchida, T. K., Habib, A., Dembia,C. L., Dunne, J. J., Ong, C. F., DeMers, M. S., Ra-jagopal, A., Millard, M., et al. (2018). OpenSim: Sim-ulating musculoskeletal dynamics and neuromuscularcontrol to study human and animal movement. PLoSComputational Biology, 14(7):1–20.

[2] Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers,J., and Davis, J. (2005). SCAPE: Shape Completionand Animation of People. InACM Transactions onGraphics, volume 24, pages 408–416.

[3] Nitschke, M., Dorschky, E., Heinrich, D., Schlarb, H., Eskofier, B. M., Koelewijn, A. D., and Van den Bogert, A. J. (2020). Efficient trajectory optimization for curved running using a 3D musculoskeletal model with implicit dynamics. Scientific Reports, 10(17655).

bash-model's People

Contributors

Stargazers

Watchers

Forkers

wangxihao antoinefalisse tbredbenner ck624 wasahaiah human2b haojianbing wangshubo90

bash-model's Issues

Can't load sto from 2020_Nitschke_RunMaD.zip.... and some questions on the result..

Hi,
Finally, I let BASH-Model run in my pc.
Since it has an error , as mentioned in my previous issue, I modify that line into OpenSim::TimeSeriesTable stoTable = OpenSim::TimeSeriesTable(filepath);
I downloaded 2020_Nitschke_RunMaD.zip to test the program.
I can't load any sto to run it... I only can run the osim with mot file....

Besides, without the sto, I got the result below.

There are some cyan color lines near the arms. Do they represent the arms from the OpenSim skeleton (runMaD.osim)?
Does it mean the skeleton can't be under the SCAPE surface model?

Issues on the versions of reading sto and the sto files in 2020_Nitschke_RunMaD.zip

I finally let the program work with Muscle Activation.
It has an error if you are not using OpenSim 4.0 API.
Change the code to OpenSim::TimeSeriesTable stoTable = OpenSim::TimeSeriesTable(filepath); may fix the error in OpenSim 4.1+ API.
The *.sto files from 2020_Nitschke_RunMaD.zip don't work in the program. The reason behind it may be the problem of format.....???
I open the *.sto using excel. The *.sto file from 2020_Nitschke_RunMaD.zip look like the following figure.

I modified the *.sto file and the program works well with Muscle Activation...

markers.obj

Hello,

I am trying to use your package with some of my data.
I managed to build everything on windows (I think), and I am now loading my data.
I get the following error: Error: No corresponding marker was found: AnkL @ line 180 in file: c:\users\u0101727\documents\myrepositories\bash-model\src\model.cpp, which is expected since there is no such marker in my model. My question is therefore the following: can I use a different marker set with your package? I see that the markers of the baseline model are defined in markers.obj, so I could potentially adjust this file. But I don't know where the values come from. I can also try to adjust my marker set to reflect what is in markers.obj. Any thoughts on that?

I have also tried loading the data provided with Nitschke et al. (2020), and I think it worked fine. At least I get a lot of .ply files in cache\mesh\.... Is it what it is supposed to output? How are you then converting those files into animation as in your example? Sorry for the basic questions, but it is not super clear to me how to use the tool and what to expect as output.

Thanks in advance for your help, I hope I can make it work - it looks great!

Prebuilt SCAPE x64 binaries?

Hi,

I came across your project which looks really interesting. Unfortunately, getting SCAPE to compile correctly is quite a challenge. Do you have any prebuilt binaries for x64 you are able to share?

Regards

How to get SCAPE.obj？

Hello,
I've successfully run your project. I am trying to use your package with some of my data.
This baseline model is generic, and I don't know if it's necessary to modify it, but I'd like to know how SCAPE.obj got it. I know that the underlying articulated skeleton and the attached marker points are modeled in Blender, but I want to know how the surface mesh inside this is obtained.
Thanks in advance for your help!

Need help for the installation........

Hello,
For the dependencies, SuitSparse package: suitsparse + amd + umfpack (>= 5.1.2).
The official SuitSparse doesn't support Windows well.
Are you using the unoffical SuitSparse for windows version?
There is a "libmwlapack.lib" I failed to get it... Does it come from the matlab version of lapack?
I am using the SDK from OpenSim 4.3 binary version.
There is an error in the code in osim.cpp....
Severity Code Description Project File Line Suppression State Error C2440 'initializing': cannot convert from 'OpenSim::DataAdapter::OutputTables' to 'OpenSim::TimeSeriesTable_<SimTK::Real>' SCAPE E:\innovision2021\BASH-Model-main\BASH-Model-main\src\osim.cpp 94

gsl.lib windows

Hello,

I'd like to test your tool but I'm struggling to get things built on windows. I was just wondering if that's something you tried out and, if so, you found an easy way to get gsl.lib? Your tool looks really great so I am really willing to give it a try.

Thanks

FOV in deg instead of rad.

Hi,

It is not an issue per se, but I think the FOV set here is in degrees, whereas glm::perspective expects a FOV in radian here. Interestingly, it does not have a huge effect on the rendering. Yet the values of the projection matrix P are wrong should anyone would like to use them. Thought that could be useful.

Best,
Antoine

How to define the markers set in SCAPE?

Hello,
I am learning about OpenSim and SMPL model. I read another issue markers.obj.

We placed the markers on the surface of the baseline model's mesh by hand and exported their coordinates

Explore the model in Blender, find the positions on the skin to place the markers that share the same name in your osim model?
How to evaluate the markers in Blender?
Thank you.

How to obtain the osim file?

Thank you very much for your perfect work. I'm new to this field, so I want to ask some basic questions. How to obtain the visualization like the gif? When I run the start.bat on windows, it said that I need to drag the .osim file. I know the .osim file is used for OpenSim, but is there any ready-made .osim file for visualization? In details, How to obtain the osim file by ourselves?

Thank you very much.