Topic: llm-serving Goto Github
Some thing interesting about llm-serving
Some thing interesting about llm-serving
llm-serving,RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Organization: alibaba
llm-serving,A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
User: asprenger
llm-serving,It is a comprehensive resource hub compiling all LLM papers accepted at the International Conference on Learning Representations (ICLR) in 2024.
User: azminewasi
llm-serving,Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.
Organization: bentoml
Home Page: https://bentoml.com
llm-serving,A guide on how to run LLMs on intel CPUs
User: biosfood
llm-serving,Lightweight and extensible LLM Inference serving benchmark tool written in Rust.
Organization: centml
llm-serving,🪶 Lightweight OpenAI drop-in replacement for Kubernetes
User: chenhunghan
llm-serving,A self-hosted personal chatbot API with FastAPI. It allows you to interact with the Llama2 LLM (and other open-source LLMs) to have natural language conversations, generate text, and perform various language-related tasks.
User: ehsanghaffar
llm-serving,A Production-Ready, Scalable RAG-powered LLM-based Context-Aware QA App
User: fork123aniket
llm-serving,Friendli: the fastest serving engine for generative AI
Organization: friendliai
Home Page: https://friendli.ai
llm-serving,Streaming of LLM responses in realtime using Fastapi and Streamlit.
User: george-mountain
llm-serving,Building Static Web Applications using Large Language Model. From hand sketched documents, images and screenshots to proper web pages.
User: george-mountain
llm-serving,Efficient AI Inference & Serving
Organization: hpcaitech
Home Page: https://hpc-ai.com/
llm-serving,Automating the deployment of the Takeoff Server on AWS for LLMs
User: inquestgeronimo
llm-serving,internet llm - access your ollama (or any other local llm) instance from across the internet
User: ivynya
llm-serving,本项目旨在分享大模型相关技术原理以及实战经验。
User: liguodongiot
Home Page: https://www.zhihu.com/column/c_1456193767213043713
llm-serving,Deep learning environment setups
User: liux2
llm-serving,Hinglish Chatbot powered by Azure Cognitive Services, Google Translate and Open AI
User: loopglitch26
Home Page: https://hinglish-chatbot-loopglitch26.streamlit.app/
llm-serving,A collection of all available inference solutions for the LLMs
User: mani-kantap
llm-serving,This repository demonstrates LLM execution on CPUs using packages like llamafile, emphasizing low-latency, high-throughput, and cost-effective benefits for inference and serving.
User: mddunlap924
llm-serving,AICI: Prompts as (Wasm) Programs
Organization: microsoft
llm-serving,A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine
Organization: mosecorg
Home Page: https://mosec.readthedocs.io/
llm-serving,Awesome-LLM-Productization: a curated list of tools/tricks/news/regulations about AI and Large Language Model (LLM) productization
Organization: oscinis-com
llm-serving,A REST API for vLLM, production ready
Organization: oss-pole-emploi
Home Page: https://oss-pole-emploi.github.io/happy_vllm/
llm-serving,Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Organization: predibase
Home Page: https://loraexchange.ai
llm-serving,Ray and Anyscale for UC Berkeley AI Hackathon!
Organization: ray-project
Home Page: https://ai.calhacks.io/
llm-serving,
Organization: ray-project
llm-serving,Deploy and Scale LLM-based applications
Organization: ray-project
Home Page: https://home.mlops.community/public/events/ray-workshop-2023-06-15
llm-serving,Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Organization: ray-project
Home Page: https://ray.io
llm-serving,This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.
Organization: ray-project
llm-serving,RayLLM - LLMs on Ray
Organization: ray-project
Home Page: https://aviary.anyscale.com
llm-serving,LLM (Large Language Model) FineTuning
User: rohan-paul
llm-serving,SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed execution—all with a simple interface.
Organization: skypilot-org
Home Page: https://skypilot.readthedocs.io
llm-serving,Run GPU inference and training jobs on serverless infrastructure that scales with you.
Organization: slai-labs
Home Page: https://beam.cloud
llm-serving,
User: stosan
Home Page: https://code-commentator.onrender.com/
llm-serving,Finetune LLMs on K8s by using Runbooks
Organization: substratusai
Home Page: https://www.substratus.ai
llm-serving,npm like package ecosystem for Prompts 🤖
Organization: sugarcane-ai
Home Page: https://sugarcaneai.dev
llm-serving,
Organization: sugarcane-ai
Home Page: https://sugarcaneai.dev
llm-serving,You can run any large language model on your local machine with this repository.
User: suleymansevimli
llm-serving,🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.
Organization: superduperdb
Home Page: https://superduperdb.com
llm-serving,A high-throughput and memory-efficient inference and serving engine for LLMs
Organization: vllm-project
Home Page: https://docs.vllm.ai
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.