Comments (12)
@ZhiGaomcislab @RongKaiWeskerMA Hi, thanks for raising this issue. My guess is that the problem is caused by --dim
argument which is 64
by default and we used 1600
in the paper. We apologize if we haven't mentioned it in the paper, this is the default value used by many few-shot learning papers. Concretely, we are able to reproduce the results on the 1-shot 5-way MiniImageNet task with the following command:
python train_protonet.py --dataset MiniImageNet --way 30 --shot 1 --lr 0.005 --step 80 --gamma 0.5 --c 0.01 --model convnet --hyperbolic --not-riemannian --dim 1600
where the parameters are as reported in the paper (--not-riemannian
is a dummy argument to be removed soon) and note that for training --way
is set to 30 as commonly done.
Please tell me if this helps!
We will add the training scripts for complete reproducibility to the repo asap.
from hyperbolic-image-embeddings.
Do you use exactly the same hyperparameters in their paper?
from hyperbolic-image-embeddings.
I've tried all the hyperparameters reported in the paper, however, got 40%
from hyperbolic-image-embeddings.
I've tried all the hyperparameters reported in the paper, however, got 40%
Yes, I use the mentioned hyperparameters in the paper, and the final results are about 40%.
from hyperbolic-image-embeddings.
Thanks @KhrulkovV a lot, the performance improved a lot for 5 way 1 shot testing.
Can I ask a question that for 5 way testing, why would we use 30 way in training phase instead?
In terms of the result for 5 way 1 shot MiniImageNet task, the final test accuracy is around 51% which is slightly off with 54% with 4 layers ConvNet, how can I resolve this issue?
from hyperbolic-image-embeddings.
@ZhiGaomcislab @RongKaiWeskerMA Hi, thanks for raising this issue. My guess is that the problem is caused by
--dim
argument which is64
by default and we used1600
in the paper. We apologize if we haven't mentioned it in the paper, this is the default value used by many few-shot learning papers. Concretely, we are able to reproduce the results on the 1-shot 5-way MiniImageNet task with the following command:
python train_protonet.py --dataset MiniImageNet --way 30 --shot 1 --lr 0.005 --step 80 --gamma 0.5 --c 0.01 --model convnet --hyperbolic --not-riemannian --dim 1600
where the parameters are as reported in the paper (--not-riemannian
is a dummy argument to be removed soon) and note that for training--way
is set to 30 as commonly done.
Please tell me if this helps!We will add the training scripts for complete reproducibility to the repo asap.
What are the values of dim and way for ResNet-18?
from hyperbolic-image-embeddings.
MINI-R18-1SHOT-5WAY.log
I have tried to run the same for ResNet-18 on the MiniImageNet dataset. The results I obtained is around 29%. I have tried for 30 Way too .. but the results are same.
Can you please help regarding this?
from hyperbolic-image-embeddings.
@sumo8291 Hey, sorry for the late response, I will look into it tomorrow.
from hyperbolic-image-embeddings.
MINI-R18-1SHOT-5WAY.log
I have tried to run the same for ResNet-18 on the MiniImageNet dataset. The results I obtained is around 29%. I have tried for 30 Way too .. but the results are same.Can you please help regarding this?
I have used the Pretrained models https://drive.google.com/drive/folders/19TdjthkqMKLKSVHrbT5pVEmKvu-6-6iM and have achieved 56.2% accuracy on Resnet18 for 1-SHOT and 30WAY.
from hyperbolic-image-embeddings.
Hello!
The following script should be able to obtain the result from the paper for resnet18 miniImageNet 1-shot task (using the reported params from the paper).
python train_protonet.py --dataset MiniImageNet --way 30 --shot 1 --lr 0.001 --step 80 --gamma 0.5 --c 0.01 --model resnet18 --hyperbolic --not-riemannian --dim 512
from hyperbolic-image-embeddings.
@sumo8291 @RongKaiWeskerMA @ZhiGaomcislab hey, we've uploaded the scripts to reproduce all the results to https://github.com/leymir/hyperbolic-image-embeddings/tree/master/examples/fewshot/configs
Closing now.
from hyperbolic-image-embeddings.
Hello, sorry to follow up on this closed issue, but I'm having problems reproducing the results, even with the new scripts. I have not modified anything -- I literally just ran the scripts.
Here are is what I ran and what I got for CUB 1s5w (conv4):
# CUB 1s5w (conv4) 0.6402 +- 0.002
python train_protonet.py \
--dataset CUB \
--shot 1 \
--lr 0.001 \
--step 50 \
--gamma 0.8 \
--c 0.05 \
--model convnet \
--hyperbolic \
--not-riemannian \
--dim 1600
### OUTPUT ###
batch 9998: 62.48(49.33)
batch 9999: 62.47(54.67)
batch 10000: 62.47(54.67)
Val Best Acc 0.6809, Test Acc 0.6247
Test Acc 0.6247 + 0.0024
Here are is what I ran and what I got for MiniImageNet 1s5w (conv4):
# MiniImageNet 1s5w (conv4) 0.5443 +- 0.002
python train_protonet.py \
--dataset MiniImageNet \
--way 30 \
--shot 1 \
--lr 0.005 \
--step 80 \
--gamma 0.5 \
--c 0.01 \
--model convnet \
--hyperbolic \
--not-riemannian \
--dim 1600
### OUTPUT ###
batch 9998: 52.15(44.00)
batch 9999: 52.15(60.00)
batch 10000: 52.15(42.67)
Val Best Acc 0.5362, Test Acc 0.5215
Test Acc 0.5215 + 0.0020
from hyperbolic-image-embeddings.
Related Issues (14)
- A typo on the main plot and poincare_mean implementation HOT 1
- Code to calculate the hyperbolicity of image datasets HOT 3
- Is it necessary to use Riemannian optimzers?
- Calculating Hyperbolicity bug in delta.py? HOT 1
- mnist example get loss nan HOT 5
- Visualization of the feature embeddings in the hyperbolic space HOT 3
- is it a bug in HyperbolicMLR?
- Can't achieve the accuracy as reported in the paper. HOT 2
- Poincare distance to origin HOT 3
- Hyperbolic CNN HOT 2
- Reproducing Results
- could you release the code for reid๏ผ
- The values of '_mobius_add(-x, y, c).norm(dim=-1, p=2, keepdim=keepdim)' are ones HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hyperbolic-image-embeddings.