Comments (8)
Did you set an excessive max_token
value? If the value is too large it may take a huge amount of time to calculate, for instant response I recommend setting it to 32 or less.
from fauxpilot.
I would also add that you should use stop tokens to stop generation when you've generated enough. This is one of the biggest "tricks" Copilot uses; all of their requests are 512 tokens but due to stop tokens they almost never actually generate that many.
from fauxpilot.
If you go to Advanced -> Edit in settings.json
in the Copilot VSCode settings, you should be able to see what extra settings are available. I think the one you want is length
:
from fauxpilot.
Thanks for the replies. These are really helpful. And if I am using the copilot plugin where should I put these configs?
from fauxpilot.
Here is a request generated from vs code, it seems that the stop
is already added. But the speed is still very slow.
from fauxpilot.
My 3090 GPU is going to be crazy when I set max_tokens to 100, and it would take minutes to get suggestions appear. Just curious about the capability of your GPU.
from fauxpilot.
My 3090 GPU is going to be crazy when I set max_tokens to 100, and it would take minutes to get suggestions appear. Just curious about the capability of your GPU.
You can see my configs in the issue and with the default config of max_tokens = 500
and the codegen-350M-multi
model, it may take more than 3 - 4 seconds to get results...But the response time is kind of random... sometimes the result returns fast.
BTW, how can I change the max_tokens
in VS Code copilot plugin?
from fauxpilot.
Can I set this max_token on fauxpilot side? I'm using the copilot plugin on vim and didn't find a way to configure this.
from fauxpilot.
Related Issues (20)
- CodeT5+ as the next model for FauxPilot? HOT 2
- can I launch fauxpilot without docker installation in notebook? HOT 1
- could Fauxpilot help to generate unit test for java code?
- Is it normal so much time to build? HOT 4
- Infinite time to (load?) and then it doen't even work?? HOT 4
- [bug] docker(version 20.10.21) version parse error in launch.sh HOT 1
- Support arm64 to minimize cost
- Maybe add windows/etc installer all-in-one in this project's 'releases'.
- 400 Bad Request when file has around 100 lines of code HOT 3
- C# support! HOT 2
- Hello all. The comments above have been very helpful in setting up the Copilot extension. I managed to get it to work with my instance and figured I would combine the steps I used (this is for Windows. Linux installation is similar, just different locations):
- It was working fine before... HOT 1
- Support for AMD GPUs HOT 1
- Triton doesnt exist anymore I think? HOT 3
- K8s deployment (via helm chart) HOT 2
- Caught signal 11 (Segmentation fault: address not mapped to object at address (nil)) HOT 1
- why my response are all !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! HOT 3
- Can I merge images of triton and client into one?eg fastertransformer_backend get content_fetch <fastertransformer&client>in CMakeLists ? HOT 1
- help me HOT 1
- What is the comparison of these model in huggingface? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fauxpilot.