Comments (4)
这里out是所有时间步的输出,默认好像就是双向输出的拼接,我只不过把前后两部分相加了。我感觉这个随便吧,keras里的Bidirectional(layer, merge_mode='concat', weights=None),merge_mode也提供了'concat', 'sum'等多种方式。
from examples.
@uhauha2929
我还有个问题,这里的HOP是什么,
embedding, attn_weights = self.attention(out.transpose(0, 1))
(B, HOP, H)
在关于维度的变化这方面我很苦恼,不知道用哪些函数,什么时候要改变下维度,请问您是怎么处理这些维度变换的。
from examples.
你可以看看这篇论文的2 APPROACH部分https://arxiv.org/abs/1703.03130。
我这里写的并不是完全按照论文里的公式,我是直接将隐层向量经过非线性层映射到一个数值,然后再softmax计算权重,最后再将所有outputs按照权重加权求和。
我是这样理解的:
论文中公式(5)他乘了个w,认为是一个HOP,公式(6)乘了个W,认为是r个HOP,实际上他用了两个MLP才将隐层向量映射成一个数值,因为他想让不同的attention集中在句子的不同的地方,但是实际中我发现这样有冗余,论文中是尝试通过加入惩罚项解决。
所以我这里的(B, HOP, H)注释不太对。
维度基本上常用的就是那几个函数,例如:transpose(0, 1)、t() 交换维度、转置;删除维度为1的维度squeeze(0)、添加一个维度unsqueeze(0);维度重排permute(1, 0, 2);变换维度view(batch_size, -1)
from examples.
能共享一下csv数据文件吗?谢谢
from examples.
Related Issues (4)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from examples.