Comments (2)
rick-github
commented on July 17, 2024
The files that ollama (actually llama.cpp) uses are in GGUF format, different from the format of the files used in transformers. There is some support for GGUF in transformers and you need to specify it's a GGUF file with gguf_file. Note it only does a few architectures, so your downloaded models may not work and you might have to try a different one.
from ollama.
RaoPisay
commented on July 17, 2024
Thank you for the clarification; I had been searching extensively online to find a method that aligned with my needs
I appreciate that Ollama downloads the entire LLM model, although I've encountered challenges in utilizing it as intended. Perhaps exploring enhancements could further optimize its usability.
from ollama.
Related Issues (20)
- Multiple windows instances with different ports HOT 1
- erorr loading models x3 7900 XTX HOT 4
- Wrong version in UI with custom build HOT 3
- Allow using `"""` in TEMPLATE Modelfile command
- GPU isn't detected in Docker WSL2 in Win11
- When I use the GLM4 model, the return result is garbled. HOT 1
- ollama-docker-app using 100% without reason in idle state HOT 1
- Environment variable OLLAMA_NUM_PARALLEL is ignored (Linux)
- Is ollama since 0.2.1 slower on CPU's HOT 1
- Avoid blocking requests to already loaded models while loading another model HOT 1
- Mistral Codestral Mamba 7B HOT 1
- Prompt Tokens for Image Chat
- SmolLM family
- Installation on Linux fails because /usr/share/ollama does not exist. HOT 4
- How to Set Up RAG / LLamaIndex with Windows Preview?
- bug: Open WebUI RAG Malfunction with Ollama Versions Post 0.2.1 HOT 4
- Releases page: please also generate an archive with dependencies
- How can I make the model produce consistent and stable results for the same prompt?
- support minicpm language model
- ROCm Memory Issues with Long Contexts HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ollama.