Comments (2)
Yeah, I noticed this issue, happend due to loading tokenizer inside the function here
https://github.com/arc53/DocsGPT/blob/main/application/utils.py
Now I load it outside. Try git pulling, difference should be extremely marginal after the update, if anything we can try using smaller tokenizer as in our ingestion function or including an setting in your settings core file
from docsgpt.
Yeah, I noticed this issue, happend due to loading tokenizer inside the function here https://github.com/arc53/DocsGPT/blob/main/application/utils.py Now I load it outside. Try git pulling, difference should be extremely marginal after the update, if anything we can try using smaller tokenizer as in our ingestion function or including an setting in your settings core file
Thanks, will give it a try soon.
from docsgpt.
Related Issues (20)
- 🐛 Bug Report: Source Doc Upload/Training
- 🐛 Bug Report: HOT 1
- fix: input box
- 🐛 Bug Report: UI fixes HOT 1
- Feature: Add graph retriever
- Feature: Set default i18n language depending on browsers language HOT 2
- [FEAT] After a successful upload, source doc should automatically switch to the new one. HOT 2
- Improvement: Changes required in the DocsGPT Docs.
- 🐛 Bug Report: Build fails after importing DocsGPTWidget in React HOT 1
- 🐛 Bug Report: Can not Run the Backend HOT 2
- Retry button on failed completion HOT 1
- Using Oracle Vector store HOT 2
- Update nextra docs with new html code block installation instruction HOT 2
- Use denominations on tokens
- Translation - Japanese - i18n HOT 4
- Translation - Mandarin - i18n HOT 2
- 🐛 Bug Report: ImportError: cannot import name 'Celery' from partially initialized module 'celery' HOT 2
- 🐛 Bug Report: Language switch to Japanese or Mandarin, blackouts the Settings tab selection HOT 2
- 🐛 Bug Report: Fix modals
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docsgpt.