Comments (6)
Hi, it should be more or less linear since it is applied on GPT-2, which is a sequential model, and the method repeats the same operation until all keywords are generated.
If the keywords are very different from each other such that it is harder to generate a sentence that contains them it may take a bit longer since it will need more conditioned generation steps, and those are more time-consuming that the unconditioned generation ones. But on average this delay should not be very relevant.
Hope this helps.
from k2t.
Hi,
it is expected with the current implementation of the code, which is not optimized for inference speed. The main bottleneck is the computation of the cosine distances, which is done in each iteration in lines 129 and 140 in main. If instead you pre-compute those distances, which you can do since you give in advance the keywords you want to guide towards, the method will be much faster. I did some experiments with that but did not have time to make a proper update, it should speed up by 5-10x.
Pre=computing, basically means calculating the cosine distances in main, e.g. line 741, storing those distances in a large vector and passing it to the conditional_language_generation function, which then does not need to re-compute it at every iteration.
I hope this helps, let me know if it does :)
from k2t.
Hi, I did it already actually (https://github.com/cahya-wirawan/K2T/tree/precalc_cosine). It is just a simple update of the cosine calculation. It is actually not really precalculated cosine similarity, but the the cosine of spec keyword is calculated only once during inference. Following the comparison of the inference time:
- data/keyword_to_articles/test_8.txt: original: 1090.27s, precalc_cosine: 185.16s (5.88X)
- data/keyword_to_articles/test_5.txt: original: 215.39s, precalc_cosine: 36.06s (5.97X)
- data/keyword_to_articles/test_10.txt: original: 498.69s, precalc_cosine: 103.86s (4.8X)
which is a nice inference time improvement, but still they are quite slow, since it still call sample_sentence() a lot of time.
from k2t.
Yes, one limitation of the method as of now is that it's slow, it should still be optimized for inference speed. I'm not sure which configuration you are using, but if you are using beam search, the quality of the generated text is better but it takes considerably longer than using just one beam and nucleus sampling.
from k2t.
Hi, I have a quick question. Is this inference time linearly related to the number of keywords? I find that the inference time reaches 10,000 seconds with about 1K keywords (100 words generated). But if I use only 5 keywords the inference time is around 50 seconds.
from k2t.
Thanks for your reply! If I understand correctly, K2T seems to be better suited for a smaller bag of words (e.g., 5 tokens), right? For example, if I want to generate text on a certain topic, instead of using all words related to the topic as keywords, would it be more appropriate to select a few representative words as keywords for the K2T generation strategy?
from k2t.
Related Issues (5)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from k2t.