Comments (11)
I am figuring it out. I will contact you as soon as possible.
from intergen.
Sorry for that, there are some typos in evaluator.py. We have already fixed that.
please make sure your code is up to date.
from intergen.
Thanks for reply. I am sure my code is up to date.
Can you release the checkpoint that exclusively trained on the training set? That would be really helpful.
from intergen.
I have trained for 1500 epochs with a batch size of 16 and I have a 12.9409 in FID compared to the 5.9 reported in the paper. Is there any reason for such a difference? All the rest of the parameters in the configs files were the ones used in the training of the model reported in the paper?
Thanks :)
from intergen.
@tr3e Any news on the issue? I have trained a model with same configuration as the one in your repo (except the batch size)
GENERAL:
EXP_NAME: IG-S-8
CHECKPOINT: ./checkpoints
LOG_DIR: ./log
TRAIN:
LR: 1e-4
WEIGHT_DECAY: 0.00002
BATCH_SIZE: 16
EPOCH: 2000
STEP: 1000000
LOG_STEPS: 10
SAVE_STEPS: 20000
SAVE_EPOCH: 100
RESUME: #checkpoints/IG-S/8/model/epoch=99-step=17600.ckpt
NUM_WORKERS: 2
MODE: finetune
LAST_EPOCH: 0
LAST_ITER: 0
But these are my results using your evaluation script:
========== MM Distance Summary ==========
---> [ground truth] Mean: 3.7844 CInterval: 0.0012
---> [InterGen] Mean: 3.8818 CInterval: 0.0017
========== R_precision Summary ==========
---> [ground truth](top 1) Mean: 0.4306 CInt: 0.0070;(top 2) Mean: 0.6110 CInt: 0.0086;(top 3) Mean: 0.7092 CInt: 0.0060;
---> [InterGen](top 1) Mean: 0.2517 CInt: 0.0071;(top 2) Mean: 0.3818 CInt: 0.0048;(top 3) Mean: 0.4662 CInt: 0.0046;
========== FID Summary ==========
---> [ground truth] Mean: 0.2966 CInterval: 0.0085
---> [InterGen] Mean: 10.7803 CInterval: 0.1791
========== Diversity Summary ==========
---> [ground truth] Mean: 7.7673 CInterval: 0.0440
---> [InterGen] Mean: 7.8075 CInterval: 0.0274
========== MultiModality Summary ==========
---> [InterGen] Mean: 1.5340 CInterval: 0.0615
As you can observe, the results are very distant from the ones provided in the paper. I am in an ongoing research using your dataset, but in order to make a fair comparison, we need to be able to replicate your results.
Hope you find what's going on :)
from intergen.
Hello!
I have run the newest training code exactly in this repo with a batch size of 64 (32 for each of 2 GPUs) for 1500 epochs.
The results are like this:
========== MM Distance Summary ==========
---> [ground truth] Mean: 3.7847 CInterval: 0.0007
---> [InterGen] Mean: 4.1817 CInterval: 0.0009
========== R_precision Summary ==========
---> [ground truth](top 1) Mean: 0.4248 CInt: 0.0046;(top 2) Mean: 0.6036 CInt: 0.0044;(top 3) Mean: 0.7026 CInt: 0.0047;
---> [InterGen](top 1) Mean: 0.3785 CInt: 0.0052;(top 2) Mean: 0.5163 CInt: 0.0040;(top 3) Mean: 0.6350 CInt: 0.0032;
========== FID Summary ==========
---> [ground truth] Mean: 0.2981 CInterval: 0.0057
---> [InterGen] Mean: 5.8447 CInterval: 0.0735
========== Diversity Summary ==========
---> [ground truth] Mean: 7.7516 CInterval: 0.0163
---> [InterGen] Mean: 7.8750 CInterval: 0.0324
========== MultiModality Summary ==========
---> [InterGen] Mean: 1.5634 CInterval: 0.0334
We suggest that you can update to the newest code, and kindly increase the batch size.
from intergen.
@tr3e I am still unable to replicate the results. Can you provide me with some contact method to talk with you and not fill this issue?
from intergen.
@tr3e I am still unable to replicate the results. Can you provide me with some contact method to talk with you and not fill this issue?
me too.
from intergen.
my email is [email protected] :)
from intergen.
Hello! I have run the newest training code exactly in this repo with a batch size of 64 (32 for each of 2 GPUs) for 1500 epochs. The results are like this:
========== MM Distance Summary ========== ---> [ground truth] Mean: 3.7847 CInterval: 0.0007 ---> [InterGen] Mean: 4.1817 CInterval: 0.0009 ========== R_precision Summary ========== ---> [ground truth](top 1) Mean: 0.4248 CInt: 0.0046;(top 2) Mean: 0.6036 CInt: 0.0044;(top 3) Mean: 0.7026 CInt: 0.0047; ---> [InterGen](top 1) Mean: 0.3785 CInt: 0.0052;(top 2) Mean: 0.5163 CInt: 0.0040;(top 3) Mean: 0.6350 CInt: 0.0032; ========== FID Summary ========== ---> [ground truth] Mean: 0.2981 CInterval: 0.0057 ---> [InterGen] Mean: 5.8447 CInterval: 0.0735 ========== Diversity Summary ========== ---> [ground truth] Mean: 7.7516 CInterval: 0.0163 ---> [InterGen] Mean: 7.8750 CInterval: 0.0324 ========== MultiModality Summary ========== ---> [InterGen] Mean: 1.5634 CInterval: 0.0334
We suggest that you can update to the newest code, and kindly increase the batch size.
Hi, I found that the MMDist here is lower than what is presented in the paper. When I am reproducing your work as well as my model, this MMDist is always around 4. Is there any mistake in the calculation?
from intergen.
The R_precision of InterGen that I reproduced is always higher than that of GT. Does anyone know the reason for this? Thank you very much.
from intergen.
Related Issues (20)
- Missing 3 joints from SMPL HOT 1
- Person-to-person generation.
- pip install -r requirements.txt failed
- Question for input data HOT 1
- How to visualize the generated results with mesh? HOT 1
- About converting "motions" to "motions_processed".
- Weird visualization from 6D rotation representation. HOT 2
- intergen dataset
- latest evaluator.py HOT 1
- The specific composition of 492 columns in each npy file HOT 2
- Global_mean.npy and global_std.npy calculations
- WRONG 6d rotation params of motion_processed about joint 20(left_hand) and 21(right_hand) HOT 7
- Metric emb scale
- World Frame Determination HOT 1
- Request for Intergen Experiment Source Code
- How to process SMPL to 6D representation
- Recovered Mesh from Rotation in processed_motion
- Resulting R_precision accuracy is always higher than the ground truth HOT 1
- Question about the SMPL-H skeleton name of npy file in motion_processed?
- [training.py] Missing Required Data Files: 'ignore_list.txt' and 'train.txt' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from intergen.