Comments (3)
GDP-no-D
commented on September 13, 2024
1
Q = self.fc_Q(x) #[b,s,head * d_head]
K = self.fc_K(x)
V = self.fc_V(x)
Q = Q.view(batch_size * self.num_head, -1, self.dim_head)
K = K.view(batch_size * self.num_head, -1, self.dim_head)
V = V.view(batch_size * self.num_head, -1, self.dim_head)
在view Q,K,V的时候就有点问题吧，应该是先把最后一个维度split开来,再交换维度,把head弄到第二维，后续再对每一个head做atttention. 这里的view我试了下顺序会乱掉啊
from chinese-text-classification-pytorch.
wjczf123
commented on September 13, 2024
对，是这样的，应该有点问题
from chinese-text-classification-pytorch.
nutshell999
commented on September 13, 2024
各位是怎么修改的啊？我也在用，而且他的位置嵌入是二维的，我的输入是三维的。
from chinese-text-classification-pytorch.
Related Issues (20)
- TextRCNN运行时报错
- Text_RCNN文本分类效果
- 使用自己的数据集出现问题 HOT 1
- FastText的n_gram_vocab HOT 1
- 一个文本要是有多个标签要怎么用这个模型呢 HOT 3
- 用模型进行预测，该怎么实现呢 HOT 2
- 如果想改成英文数据，都需要修改什么呢
- transformer HOT 2
- 可以写一个单条预测的接口嘛？
- RuntimeError: GET was unable to find an engine to execute this computation
- 是否有文本长度限制 HOT 1
- Bi-LSTM问题
- 是否是没有进行分词？
- 选择题如何定制输入
- 词向量训练问题
- [BUG] TextCNN训练速度很慢
- 请问为何TextCNN的一个iter为什么要十几分钟，而TextRNN就很快，一个epoch不到一分钟？ HOT 2
- fasttext 训练好的模型转onnx
- dpcnn某个地方代码有问题
- 英文的文本分类怎么用
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chinese-text-classification-pytorch.