Comments (8)
AaronJackson
commented on June 12, 2024
If the input image has been scaled by VRN then the output will need to be scale back to the original by applying the inverse transform. There should not be any need to apply any manual scaling to bring things back into alignment. ICP should only need to be used for finding the correspondence, once you have that the NME can be calculated
from vrn.
ydwen
commented on June 12, 2024
Sorry for the confusion.
My actual question is how to associate the xyz coordinates provided by the obj file, and those in the rendered face image. Aren't they in different scales? Is it addressed by ICP?
Thanks.
from vrn.
AaronJackson
commented on June 12, 2024
The input image is scaled using a landmark detector, so the vertices may have a different scale to the original input image. The inverse of this can be applied to go back to the original image scale. To figure out what this scale is, you'll need to go through the code and see how it is calculated.
I don't really understand your question to be honest, sorry
from vrn.
ydwen
commented on June 12, 2024
Really sorry for my bad description. But we are getting close to it.
We have two things provided by the florenceface dataset.
a. The original images. (or rendered images?)
b. The xyz coordinates of the vertices.
From the original image, VRN predicts xyz coordinates. They are in different scales, right?
from vrn.
AaronJackson
commented on June 12, 2024
Like I said, some scale is applied to normalise the face to an expected
size. You'll need to apply the inverse of this scaling to go back to the
original size. This scaling is calculated and performed lines 43-62 of
`run.m`. Apply the inverse of these operations in the correct order to
the predicted mesh to go back to your original scale. You do not need to
use ICP to find the scale.
Yandong Wen writes:
… Really sorry for my bad description. But we are getting close to it.
We have two things provided by the florenceface dataset.
a. The original images. (or rendered images?)
b. The xyz coordinates of the vertices.
From the original image, VRN predicts xyz coordinates. They are in different scales, right?
from vrn.
ydwen
commented on June 12, 2024
Got it. Thanks!
from vrn.
HOMGH
commented on June 12, 2024
Hi Yandong and Aaron, @ydwen @AaronJackson
I had a question regarding computing NME for 3D face reconstruction. When the output 3D mesh of an algorithm has for example 43k vertices (for the case of PRNet for example), how can we compare this with the groundtruth mesh which has for example 53k vertices(for the case of AFLW2000 dataset).
Would you please share the NME computing code you used based on above discussion?
Thank you.
from vrn.
AaronJackson
commented on June 12, 2024
Hi
I used pcregrigid in Matlab to find the correspondence between the two
meshes without applying a transformation. This required a modification
to the built-in Matlab function, which unfortunately I cannot share, as
it's copyrighted by MathWorks. To make matters worse, the newer versions
of Matlab only include a compiled version of pcregrigid, and as such, it
is not possible to make the same modification.
The modification was essentially to also return the indices used to
return the transformed mesh. Your best bet is to find a ICP
implementation which also returns the correspondence.
Sorry about that!
Aaron
Pingu writes:
… Hi Yandong and Aaron, @ydwen @AaronJackson
I had a question regarding computing NME for 3D face reconstruction. When the output 3D mesh of an algorithm has for example 43k vertices (for the case of [PRNet](https://github.com/YadiraF/PRNet) for example), how can we compare this with the groundtruth mesh which has for example 53k vertices(for the case of AFLW2000 dataset).
Would you please share the NME computing code you used based on above discussion?
Thank you.
from vrn.
Related Issues (20)
- Maybe Helpful: Ubuntu 18.04 to suxessfull running vrn Howto/run-throuth with Cuda10.1 HOT 2
- benchmark on AFLW2000-3D HOT 2
- Warp back onto original image
- API for website demo HOT 2
- ./run.sh using UBUNTU 18.04 failed. How can I solve this? Is there any thing wrong with the directory? HOT 10
- Error in Ubuntu 18.04 HOT 7
- NME HOT 1
- cannot open <2D-FAN-300W.t7> in mode HOT 1
- Are you going to release the train code? HOT 1
- I make a better one [https://github.com/WIKI2020/FacePose_pytorch]
- Running via Octave instead of Matlab HOT 6
- Error on Building thpp HOT 1
- windows 10 successful install? HOT 1
- Improve the result HOT 11
- Fail in main.lua : main.lua:52: '=' expected near 'continue' HOT 11
- Rendering Florence with 20 poses HOT 2
- Results in vrn-docker HOT 1
- Online demo HOT 2
- The face detector failed to find your face.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vrn.