Comments (3)
yangcedrus
commented on July 26, 2024
你好，这种在计算损失之前在距离上乘以一个数，就是有些方法会用的温度。
这些方法会用温度来控制距离分布的熵。乘上0.01后，pytorch实际进行交叉熵计算的是经过softmax的tensor。softmax的结果会受到温度影响，但是和不用温度结果相比，不会有数量级或者大小关系的变化，只会使得每个类分类结果的分布平缓或者尖锐。
你所做的乘以0.01，就是使用值为100的温度。至于温度对方法的影响，这个对不同的方法是不同的，因此准确率提升或者下降都是有可能的。
from libfewshot.
ch-control
commented on July 26, 2024
感谢回复，那我可不可以理解为距离的数量级对模型最终的准确率会有影响，那么假如我对距离度量进行了改进导致最终距离的数量级变小，我怎么能够确定最终模型准确率的提升是因为我的改进方法还是因为数量级的变化？
from libfewshot.
yangcedrus
commented on July 26, 2024
由于计算交叉熵之前需要先使用softmax对距离进行处理，因此乘上系数不能简单理解为改变了距离的数量级，系数最主要的所用是改变softmax的输出分布。
如果是只是对ProtoNet的欧式距离乘上这种系数，并不能称得上是对距离度量的改进，而是在距离上应用了温度这个技巧。
如果是对其他你所提出的距离度量进行分析，需要进行消融，尝试去掉温度或者换一个带相同温度的其他度量来进行消融实验，以此来验证是温度还是距离度量的作用。
深度学习中的temperature parameter
交叉熵的计算
from libfewshot.
Related Issues (20)
- 数据集划分规范问题 HOT 2
- RuntimeError: shape '[-1, 3, 16]' is invalid for input of size 30 HOT 2
- train，val和test类别问题 HOT 7
- How to use this code into open-set recognition? HOT 1
- 使用自己数据集的时候会卡在加载训练数据集这一步 HOT 1
- https://github.com/RL-VIG/LibFewShot.git
- The computer does not have cuda, and it will report an error when running. HOT 1
- 5way-5shot报错 HOT 9
- 报错：/LibFewShot/core/data/samplers.py", line 111, in __iter__ batch = torch.stack(batch).reshape(-1) RuntimeError: stack expects each tensor to be equal size, but got [16] at entry 0 and [7] at entry 2 HOT 14
- 不管怎么搭配，程序都报错啊 HOT 1
- 请问怎么可以对一张图片进行分类？ HOT 2
- 使用miniImageNet--ravi数据集，然后ACC值为什么这么低？ HOT 5
- RENet HOT 1
- renet.yaml的backbone为ViT时的报错解决 HOT 1
- 训练日志与run_test的test数据差距过大 HOT 2
- 关于deepbdc方法的咨询 HOT 2
- use_logger() got an unexpected keyword argument 'file' HOT 1
- 使用时报错too many values to unpack
- 在复现文件夹`reproduce/`中缺失部分 method 的 config，详情见图 HOT 5
- 论文复现成功后，提交pr会同意吗QAQ HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from libfewshot.