Comments (6)
It's available on the backend
from huggingface.js.
@co42 @Narsil , does HF endpoints support wait_for_model
like Inference API?
from huggingface.js.
We did not add the option yet, but it's definitively possible if needed.
from huggingface.js.
It would be nice for API compatibiltity between inference API & inference endpoints!
from huggingface.js.
@jinnovation you will now get a 503 in streaming requests with most recent version 2.6.5
With your current version, you can get a 503 if you set retry_on_error
to false
:
const response = hf.textGenerationStream({
inputs: experimental_buildLlama2Prompt([
{
role: "user",
content: "hello",
},
]),
{
retry_on_error: false
}
});
Soon the inference endpoint backend will be updated, so that a call by default with @huggingface/inference
will wait until the model is loaded (you can disable this behavior with retry_on_error: false
to handle the 503 yourself)
from huggingface.js.
Soon the inference endpoint backend will be updated, so that a call by default with
@huggingface/inference
will wait until the model is loaded
Fantastic! Thank you.
from huggingface.js.
Related Issues (20)
- Missing Type From the Inference Package HOT 1
- [Conversational] Property conversational does not exist on type HfInference HOT 3
- Safetensors sharded model inspector does not work in subdirs HOT 8
- Is 404 console.error expected for `fileExists ` HOT 4
- Sharded GGUF in subdir HOT 3
- fix `pipelineSnippet` for repos with custom pipelines HOT 2
- GGUF Sharded model metadata display might have a memory leak HOT 6
- GGUF: missing `split.no` metadata HOT 6
- [Question] What is the correct way to access commit diff results via http? HOT 1
- [ASR Widget] “Browse for file” is unresponsive HOT 1
- Inference API widget does not work anymore for token classification of POS HOT 3
- [Widget] "Model not loaded yet" error on page load
- Rm ArrayBuffer.resize polyfill when Firefox supports it by default HOT 4
- Feature Request: Get Commits List / Commits History by Repository ID using @huggingface/hub HOT 1
- [Question] Closed source model support?
- Build a chatbot with huggingFace
- CDN: Reference unbundled/unminified files directly instead of base esm/jsdeliver links?
- Add Ollama and vLLM as Options for Local Apps in Hugginface HOT 5
- feat: simple heuristic for `isTensorrtEngine` HOT 1
- Allow `replay.mp4` rendering for robotics tasks HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from huggingface.js.