Comments (13)
NumesSanguis
commented on August 27, 2024
This seems to be related to this issue: #23
Please check that issue for updates.
from facsvatar.
l-h-e
commented on August 27, 2024
Thank you for your reply .
How about question 2 ?
from facsvatar.
NumesSanguis
commented on August 27, 2024
If I remember correctly, FACSvatar is not using the other parameters. Therefore, the non-relevant columns have been removed from demo.csv to reduce file size.
If issue 1 is solved, main.py would just ignore the extra columns outputted by OpenFaceOffline.
from facsvatar.
l-h-e
commented on August 27, 2024
Thank you .
I checked they are same by outputting received message in Unity3D .
However , they are slightly difference .
OpenFaceOffline's message includes "PoseTx , Ty , Tz" and "AU01~46" , but csv message has not .
I think the result as move of avatar must be same with same movie , however avatar's moving is a bit difference . (particular mouth moving is difference)
Is it specification or my setting issue ?
from facsvatar.
NumesSanguis
commented on August 27, 2024
About issue 1, I think you've put your OpenFace .csv in to FACSvatar\modules\input_facsfromcsv\openface\default_clean\your_recording.csv, right? .csv produces by OpenFace should not be put in a folder with a name ending in _clean. edited
I'm rewriting the documentation for FACSvatar v.0.4.0, and I hope these instructions will make it a bit clearer how to use your own files: https://facsvatar.readthedocs.io/en/v0.4.0/trackers/openface.html#use-your-own-videos
OpenFaceOffline's message includes "PoseTx , Ty , Tz"
That is correct, FACSvatar only uses "pose_Rx, Ry, Rz"
and "AU01~46" , but csv message has not .
Can you describe in more detail what you mean? FACSvatar uses the same AU values as OpenFace.
I think the result as move of avatar must be same with same movie , however avatar's moving is a bit difference . (particular mouth moving is difference)
OpenFace's facial expression tracking is not perfect, so some differences are to be expected.
It would be nice to use a better FACS tracker, however I'm not aware of an open FACS tracker that performs better than OpenFace.
from facsvatar.
l-h-e
commented on August 27, 2024
Thank you for reply .
About issue 1, I think you've put your OpenFace .csv in to FACSvatar\modules\input_facsfromcsv\openface\default_clean\your_recording.csv, right? .csv produces by OpenFace should not be put in a folder with a name ending in _default.
I see.
However , the result is not difference .
Can you describe in more detail what you mean? FACSvatar uses the same AU values as OpenFace.
This is the output in Unity3D with Debug.Log(facsvatar); if I use OpenFaceOffline.
{
"timestamp": 8.072677,
"frame": 198,
"confidence": 0.875,
"pose": {
"pose_Tx": 13.690295647423422,
"pose_Ty": 114.7141430875774,
"pose_Tz": 294.95225881900109,
"pose_Rx": -0.12924998189751064,
"pose_Ry": 0.188265749507212,
"pose_Rz": -0.14415760728584548
},
"au_c": {
"AU04": 1.0,
"AU05": 0.0,
"AU06": 0.0,
"AU07": 1.0,
"AU10": 0.0,
"AU12": 0.0,
"AU14": 0.0,
"AU23": 0.0,
"AU01": 1.0,
"AU02": 0.0,
"AU09": 0.0,
"AU15": 0.0,
"AU17": 0.0,
"AU20": 0.0,
"AU25": 1.0,
"AU26": 1.0,
"AU28": 0.0,
"AU45": 1.0
},
"blendshapes": {
"Expressions_abdomExpansion_max": 0.0,
"Expressions_abdomExpansion_min": 0.0,
"Expressions_browOutVertL_max": 0.21634,
"Expressions_browOutVertL_min": 0.32781000000000005,
"Expressions_browOutVertR_max": 0.21634,
"Expressions_browOutVertR_min": 0.32781000000000005,
"Expressions_browSqueezeL_max": 0.52522,
"Expressions_browSqueezeL_min": 0.24038,
"Expressions_browSqueezeR_max": 0.52522,
"Expressions_browSqueezeR_min": 0.24038,
"Expressions_browsMidVert_max": 0.4538,
"Expressions_browsMidVert_min": 0.52522,
"Expressions_cheekSneerL_max": 0.02729,
"Expressions_cheekSneerR_max": 0.02729,
"Expressions_chestExpansion_max": 0.0,
"Expressions_chestExpansion_min": 0.0,
"Expressions_eyeClosedL_max": 0.29848,
"Expressions_deglutition_max": 0.0,
"Expressions_deglutition_min": 0.0,
"Expressions_eyeClosedL_min": 0.32605,
"Expressions_eyeClosedPressureL_max": 0.00102,
"Expressions_eyeClosedPressureL_min": 0.13042,
"Expressions_eyeClosedPressureR_max": 0.00102,
"Expressions_eyeClosedPressureR_min": 0.13042,
"Expressions_eyeClosedR_max": 0.29848,
"Expressions_eyeClosedR_min": 0.32605,
"Expressions_eyeSquintL_max": 0.05217,
"Expressions_eyeSquintL_min": 0.0,
"Expressions_eyeSquintR_max": 0.05217,
"Expressions_eyeSquintR_min": 0.0,
"Expressions_eyesHoriz_max": 0.0,
"Expressions_eyesHoriz_min": 0.15405,
"Expressions_eyesVert_max": 0.02615,
"Expressions_eyesVert_min": 0.0,
"Expressions_jawHoriz_max": 0.0,
"Expressions_jawHoriz_min": 0.0,
"Expressions_jawOut_max": 0.0,
"Expressions_jawOut_min": 0.0,
"Expressions_mouthBite_max": 0.0,
"Expressions_mouthBite_min": 0.0,
"Expressions_mouthChew_max": 0.05557,
"Expressions_mouthChew_min": 0.0,
"Expressions_mouthClosed_max": 0.12968,
"Expressions_mouthClosed_min": 0.54759,
"Expressions_mouthHoriz_max": 0.0,
"Expressions_mouthHoriz_min": 0.0,
"Expressions_mouthInflated_max": 0.0,
"Expressions_mouthInflated_min": 0.0,
"Expressions_mouthLowerOut_max": 0.0,
"Expressions_mouthLowerOut_min": 0.0,
"Expressions_mouthOpenAggr_max": 0.00102,
"Expressions_mouthOpenAggr_min": 0.00317,
"Expressions_mouthOpenHalf_max": 0.0,
"Expressions_mouthOpenLarge_max": 0.1945,
"Expressions_mouthOpenLarge_min": 0.0,
"Expressions_mouthOpenO_max": 0.0,
"Expressions_mouthOpenO_min": 0.0,
"Expressions_mouthOpenTeethClosed_max": 0.05476,
"Expressions_mouthOpenTeethClosed_min": 0.0,
"Expressions_mouthOpen_max": 0.0,
"Expressions_mouthOpen_min": 0.0,
"Expressions_mouthSmileL_max": 0.17632,
"Expressions_mouthSmileOpen2_max": 0.0,
"Expressions_mouthSmileOpen2_min": 0.0,
"Expressions_mouthSmileOpen_max": 0.00563,
"Expressions_mouthSmileOpen_min": 0.0,
"Expressions_mouthSmileR_max": 0.17632,
"Expressions_mouthSmile_max": 0.26172,
"Expressions_mouthSmile_min": 0.0,
"Expressions_nostrilsExpansion_max": 0.00732,
"Expressions_nostrilsExpansion_min": 0.0,
"Expressions_pupilsDilatation_max": 0.0,
"Expressions_pupilsDilatation_min": 0.0,
"Expressions_tongueHoriz_max": 0.0,
"Expressions_tongueHoriz_min": 0.0,
"Expressions_tongueOutPressure_max": 0.0,
"Expressions_tongueOut_max": 0.0,
"Expressions_tongueOut_min": 0.0,
"Expressions_tongueTipUp_max": 0.0,
"Expressions_tongueVert_max": 0.0,
"Expressions_tongueVert_min": 0.0
}
}However , the item of "au_c" is not include with .csv .
This is the output in Unity3D if I use "python main.py" .
{
"confidence": 0.98,
"frame": 132,
"timestamp": 4.4289999999999994,
"pose": {
"pose_Rx": -0.14850085089567011,
"pose_Ry": 0.42489385359125481,
"pose_Rz": -0.068709993021343674
},
"timestamp_utc": 15963537882826652,
"blendshapes": {
"Expressions_abdomExpansion_max": 0.0,
"Expressions_abdomExpansion_min": 0.0,
"Expressions_browOutVertL_max": 0.04829,
"Expressions_browOutVertL_min": 1.70024,
"Expressions_browOutVertR_max": 0.04829,
"Expressions_browOutVertR_min": 1.70024,
"Expressions_browSqueezeL_max": 2.78838,
"Expressions_browSqueezeL_min": 0.05366,
"Expressions_browSqueezeR_max": 2.78838,
"Expressions_browSqueezeR_min": 0.05366,
"Expressions_browsMidVert_max": 0.00543,
"Expressions_browsMidVert_min": 2.78838,
"Expressions_cheekSneerL_max": 1.92327,
"Expressions_cheekSneerR_max": 1.92327,
"Expressions_chestExpansion_max": 0.0,
"Expressions_chestExpansion_min": 0.0,
"Expressions_eyeClosedL_max": 0.02071,
"Expressions_deglutition_max": 0.0,
"Expressions_deglutition_min": 0.0,
"Expressions_eyeClosedL_min": 0.00835,
"Expressions_eyeClosedPressureL_max": 0.0067,
"Expressions_eyeClosedPressureL_min": 0.00334,
"Expressions_eyeClosedPressureR_max": 0.0067,
"Expressions_eyeClosedPressureR_min": 0.00334,
"Expressions_eyeClosedR_max": 0.02071,
"Expressions_eyeClosedR_min": 0.00835,
"Expressions_eyeSquintL_max": 0.00134,
"Expressions_eyeSquintL_min": 0.0,
"Expressions_eyeSquintR_max": 0.00134,
"Expressions_eyeSquintR_min": 0.0,
"Expressions_eyesHoriz_max": 0.0,
"Expressions_eyesHoriz_min": 0.294,
"Expressions_eyesVert_max": 0.0,
"Expressions_eyesVert_min": 0.053,
"Expressions_jawHoriz_max": 0.0,
"Expressions_jawHoriz_min": 0.0,
"Expressions_jawOut_max": 0.0,
"Expressions_jawOut_min": 0.0,
"Expressions_mouthBite_max": 0.0,
"Expressions_mouthBite_min": 0.0,
"Expressions_mouthChew_max": 0.03772,
"Expressions_mouthChew_min": 0.0,
"Expressions_mouthClosed_max": 0.05882,
"Expressions_mouthClosed_min": 0.17891,
"Expressions_mouthHoriz_max": 0.0,
"Expressions_mouthHoriz_min": 0.0,
"Expressions_mouthInflated_max": 0.0,
"Expressions_mouthInflated_min": 0.0,
"Expressions_mouthLowerOut_max": 0.0,
"Expressions_mouthLowerOut_min": 0.0174,
"Expressions_mouthOpenAggr_max": 0.0067,
"Expressions_mouthOpenAggr_min": 1.1121,
"Expressions_mouthOpenHalf_max": 0.0,
"Expressions_mouthOpenLarge_max": 0.13202,
"Expressions_mouthOpenLarge_min": 0.0,
"Expressions_mouthOpenO_max": 0.0,
"Expressions_mouthOpenO_min": 0.0,
"Expressions_mouthOpenTeethClosed_max": 0.01789,
"Expressions_mouthOpenTeethClosed_min": 0.0,
"Expressions_mouthOpen_max": 0.0,
"Expressions_mouthOpen_min": 0.0,
"Expressions_mouthSmileL_max": 0.03529,
"Expressions_mouthSmileOpen2_max": 0.0,
"Expressions_mouthSmileOpen2_min": 0.05219,
"Expressions_mouthSmileOpen_max": 0.0,
"Expressions_mouthSmileOpen_min": 0.0,
"Expressions_mouthSmileR_max": 0.03529,
"Expressions_mouthSmile_max": 0.02721,
"Expressions_mouthSmile_min": 0.23287,
"Expressions_nostrilsExpansion_max": 1.8669,
"Expressions_nostrilsExpansion_min": 0.0,
"Expressions_pupilsDilatation_max": 0.0,
"Expressions_pupilsDilatation_min": 0.0,
"Expressions_tongueHoriz_max": 0.0,
"Expressions_tongueHoriz_min": 0.0,
"Expressions_tongueOutPressure_max": 0.0,
"Expressions_tongueOut_max": 0.0,
"Expressions_tongueOut_min": 0.0,
"Expressions_tongueTipUp_max": 0.0,
"Expressions_tongueVert_max": 0.0,
"Expressions_tongueVert_min": 0.0
}
}There are a bit difference .
from facsvatar.
NumesSanguis
commented on August 27, 2024
Thank you for the detailed log!
"au_c"means classification if that AU is active or not. Basically it means that ifau_r >= 0.5then it's1, else0. This value is not used in FACSvatar."pose_Tx"is also ignored.
There is some slight difference in using OpenFaceOffline for real-time purposes, or when using the .csv. In general, the .csv is more accurate and therefore preferred when real-time is not necessary. Apparently, OpenFace does some post-processing after analysing the whole file, when it writes the results to a .csv file. This does not happen when using it in real-time.
from facsvatar.
l-h-e
commented on August 27, 2024
Thank you for reply .
I see .
and I have another question that how to use 2 model with csv input .
in default status , I can receive following message
Received message: ['openface.p0.somefile, 15965343864475856, {'confidence': 0.98, 'frame': 22, 'timestamp': 0.738, 'au_r': {'AU01': 0.09726842766153403, 'AU02': 0.034993200875877276, 'AU04': 3.0209638855816707, 'AU05': 0.006224593312018546, 'AU06': 0.0, 'AU07': 2.0982603297049134, 'AU09': 0.027994560700701818, 'AU10': 1.5865665697855156, 'AU12': 0.0, 'AU14': 0.0, 'AU15': 0.04173519116644401, 'AU17': 0.23696478524592737, 'AU20': 0.03250339956206137, 'AU23': 0.03250339956206137, 'AU25': 0.1875066408505329, 'AU26': 0.18623398961520926, 'AU45': 0.0, 'AU61': 0.268, 'AU62': 0, 'AU63': 0, 'AU64': 0.017}, 'pose': {'pose_Rx': -0.15426555630457997, 'pose_Ry': 0.4306318043153648, 'pose_Rz': -0.05582972387846044}, 'timestamp_utc': 15965343864141326}]I want to change "p0" to "p1" .
What should I do ?
from facsvatar.
NumesSanguis
commented on August 27, 2024
Do you want 2 models with the same facial expressions (1 .csv), or 2 models with different facial expressions (2 .csv)?
In case of 2 .csv, you need to follow this naming scheme:
some_name_p0.csvsome_name_p1.csv
https://github.com/NumesSanguis/FACSvatar/tree/master/modules/input_facsfromcsv/openface/default_clean
Let me give you later some more detailed instructions.
from facsvatar.
l-h-e
commented on August 27, 2024
Thank you for reply .
I used following command .
python main.py --csv_arg 2people_60fps_p1.csv --pub_ip facsvatar_bridge .
and My UnityMainThreadDispatcher's Participants state is "Users_2_models_2".
However , The woman avatar only moves .
Docker received message "Received message: ['openface.p0.2people_60fps_p1' ... " .
What should I do ?
from facsvatar.
NumesSanguis
commented on August 27, 2024
I'll get back to you, but I believe changing p1 to p* does the trick. So it would be:
python main.py --csv_arg 2people_60fps_p*.csv --pub_ip facsvatar_bridge
Then it sends the facial data of p0.csv to the 1st avatar, and p1.csv to the 2nd avatar.
from facsvatar.
l-h-e
commented on August 27, 2024
Thank you for reply .
I see.
I tried this command and I can make 2 avatars move .
python main.py --csv_arg 2people_60fps_p*.csv --pub_ip facsvatar_bridge
from facsvatar.
NumesSanguis
commented on August 27, 2024
Great that you could confirm that :)
from facsvatar.
Related Issues (20)
- How to enhance AUs from within Facsvatar? (Fix the notebook GUI) HOT 9
- Will there be a windows stand-alone for Blender? HOT 2
- Blender avatar is not receiving data. HOT 1
- Problem with face bones HOT 3
- openface_2.1.0_zeromq generated .csv data incompatibility with input_facsfromcsv HOT 5
- How to use FACSvatar system with 2 or more computers
- Can I fix fps of OpenFaceoffline ? HOT 2
- Openface License Information. No Commercial Use Without Purchase, $18k Per Year HOT 20
- How avatar's setting in Make Human (FACS Human) HOT 2
- Facsdnnfacs for dynamic AUs? HOT 2
- How should I get python output? HOT 2
- Issue with changing bone.rotation_mode to 'XYZ' HOT 4
- Background rotation in visualizing pose angles HOT 6
- Use Google's project MediaPipe Face Mesh as FACS tracker HOT 18
- Character model replacement HOT 1
- How can I save the data captured by camera through OpenFaceOffline into a csv file? HOT 1
- FACSvatar with Blender 3.3 HOT 1
- No json file found for AU07 HOT 1
- Feature: Explore using Reallusion - free 3D Character Base as human model
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from facsvatar.