Comments (3)
It is 2048 tokens (one token can represent multiple characters so this is a decent amount of text). This is also the same limit Github Copilot uses, so it mostly works, but the tokenizer used by CodeGen vs Copilot are slightly different, which sometimes causes the VSCode Copilot send a prompt that is too long for CodeGen (but would have been fine for Copilot).
I think that this can be fixed by replacing the vocab.bpe
and tokenizer.json
files in the .vscode/extensions/github.copilot-[version]/
directory (after backing them up of course) with the ones in this directory. I haven't added this to the README yet though because I haven't had time to test it out, so do let me know if it works!
from fauxpilot.
@moyix for codex the context length is 8,000 tokens. Am I missing something?
Reference: https://beta.openai.com/docs/models/codex-series-private-beta
from fauxpilot.
Yes – there are multiple Codex models. code-davinci-002
goes up to 8000 tokens; code-cushman-001
goes up to 2048. It's not known exactly which model Copilot is currently using (they changed things in the extension so it doesn't show the engine name any more), but at least when it was released they were using Cushman.
It's also possible that by this point Copilot is using a non-public model that they or OpenAI fine-tuned for them. But you should be able to verify that it uses 2048 tokens by looking at the requests Copilot sends; if you run them through the Codex tokenizer the prompt + tokens requested add up to 2048. You can also look into Copilot's extension.js
and find:
let _={maxPromptLength:2048-(0,o.getConfig)(e,o.ConfigKey.SolutionLength)
I suspect they use the smaller model because they care a lot about latency and Cushman is quite a bit faster than DaVinci.
from fauxpilot.
Related Issues (20)
- Support for StarCoder HOT 1
- CodeGen2 compatibility HOT 9
- CodeT5+ as the next model for FauxPilot? HOT 2
- can I launch fauxpilot without docker installation in notebook? HOT 1
- could Fauxpilot help to generate unit test for java code?
- Is it normal so much time to build? HOT 4
- Infinite time to (load?) and then it doen't even work?? HOT 4
- [bug] docker(version 20.10.21) version parse error in launch.sh HOT 1
- Support arm64 to minimize cost
- Maybe add windows/etc installer all-in-one in this project's 'releases'.
- 400 Bad Request when file has around 100 lines of code HOT 3
- C# support! HOT 2
- Hello all. The comments above have been very helpful in setting up the Copilot extension. I managed to get it to work with my instance and figured I would combine the steps I used (this is for Windows. Linux installation is similar, just different locations):
- It was working fine before... HOT 1
- Support for AMD GPUs HOT 1
- Triton doesnt exist anymore I think? HOT 3
- K8s deployment (via helm chart) HOT 2
- Caught signal 11 (Segmentation fault: address not mapped to object at address (nil)) HOT 1
- why my response are all !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! HOT 3
- Can I merge images of triton and client into one?eg fastertransformer_backend get content_fetch <fastertransformer&client>in CMakeLists ? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fauxpilot.