Topic: llm-inference Goto Github
Some thing interesting about llm-inference
Some thing interesting about llm-inference
llm-inference,irresponsible innovation. Try now at https://chat.dev/
Organization: anarchy-ai
Home Page: https://anarchy.ai/
llm-inference,Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and increase inference speed.
User: b4rtaz
llm-inference,The Open Serverless GPU Cloud
Organization: beam-cloud
Home Page: https://docs.beam.cloud
llm-inference,The easiest way to serve AI apps and models - Build reliable Inference APIs, LLM apps, Multi-model chains, RAG service, and much more!
Organization: bentoml
Home Page: https://bentoml.com
llm-inference,Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.
Organization: bentoml
Home Page: https://bentoml.com
llm-inference,Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.
Organization: character-ai
Home Page: https://pypi.org/project/prompt-poet/
llm-inference,Code examples and resources for DBRX, a large language model developed by Databricks
Organization: databricks
Home Page: https://www.databricks.com/
llm-inference,📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
User: deftruth
Home Page: https://github.com/DefTruth/Awesome-LLM-Inference
llm-inference,A lightweight alternative to Kubernetes for AI, simplifying container orchestration on any cloud or on-premises and accelerating AI development, training, and deployment.
Organization: dstackai
Home Page: https://dstack.ai/docs
llm-inference,LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
User: eastriverlee
llm-inference, Embedding Studio is a framework which allows you transform your Vector Database into a feature-rich Search Engine.
Organization: eulersearch
Home Page: https://embeddingstud.io/
llm-inference,Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
Organization: fasterdecoding
Home Page: https://sites.google.com/view/medusa-llm
llm-inference,Sequence Parallel Attention for Long Context LLM Model Training and Inference
User: feifeibear
llm-inference,The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models.
Organization: flagai-open
llm-inference,FlashInfer: Kernel Library for LLM Serving
Organization: flashinfer-ai
Home Page: https://flashinfer.ai
llm-inference,Pure C++ implementation of several models for real-time chatting on your computer (CPU)
User: foldl
llm-inference,LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.
User: ghimiresunil
llm-inference,Efficient AI Inference & Serving
Organization: hpcaitech
Home Page: https://hpc-ai.com/
llm-inference,Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
User: inferflow
llm-inference,⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
Organization: intel
llm-inference,An innovative library for efficient LLM inference via low-bit quantization
Organization: intel
Home Page: https://github.com/intel/neural-speed
llm-inference,LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Organization: internlm
Home Page: https://lmdeploy.readthedocs.io/en/latest/
llm-inference,LLMs and Machine Learning done easily
Organization: kenza-ai
Home Page: https://kenza-ai.github.io/sagify/
llm-inference,LLMs as Copilots for Theorem Proving in Lean
Organization: lean-dojo
Home Page: https://leandojo.org
llm-inference,20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Organization: lightning-ai
Home Page: https://lightning.ai
llm-inference,本项目旨在分享大模型相关技术原理以及实战经验。
User: liguodongiot
Home Page: https://www.zhihu.com/column/c_1456193767213043713
llm-inference,Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
User: liltom-eth
llm-inference,AICI: Prompts as (Wasm) Programs
Organization: microsoft
llm-inference,A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap
Organization: microsoft
Home Page: https://microsoft.github.io/autogen/
llm-inference,Official inference library for Mistral models
Organization: mistralai
Home Page: https://mistral.ai/
llm-inference,AI-powered cybersecurity chatbot designed to provide helpful and accurate answers to your cybersecurity-related queries and also do code analysis and scan analysis.
User: morpheuslord
llm-inference,Practical Llama 3 inference in Java
User: mukel
llm-inference,Sparsity-aware deep learning inference runtime for CPUs
Organization: neuralmagic
Home Page: https://neuralmagic.com/deepsparse/
llm-inference,GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
Organization: nomic-ai
Home Page: https://nomic.ai/gpt4all
llm-inference,Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
Organization: nvidia
llm-inference,OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
Organization: openvinotoolkit
Home Page: https://docs.openvino.ai
llm-inference,Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Organization: predibase
Home Page: https://loraexchange.ai
llm-inference,A multi-platform SwiftUI frontend for running local LLMs with Apple's MLX framework.
Organization: preternatural-explore
llm-inference,FineTune LLMs in few lines of code (Text2Text, Text2Speech, Speech2Text)
Organization: promptslab
llm-inference,This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.
Organization: ray-project
llm-inference,RayLLM - LLMs on Ray
Organization: ray-project
Home Page: https://aviary.anyscale.com
llm-inference,A tool for generating function arguments and choosing what function to call with local LLMs
User: rizerphe
Home Page: https://local-llm-function-calling.readthedocs.io/
llm-inference,LLM (Large Language Model) FineTuning
User: rohan-paul
llm-inference,GPU environment and cluster management with LLM support
Organization: run-ai
Home Page: https://www.genv.dev
llm-inference,Official Implementation of EAGLE-1 and EAGLE-2
Organization: safeailab
Home Page: https://arxiv.org/pdf/2406.16858
llm-inference,High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
Organization: sjtu-ipads
llm-inference,LLMFlows - Simple, Explicit and Transparent LLM Apps
User: stoyan-stoyanov
Home Page: https://llmflows.readthedocs.io
llm-inference,Superduper: Bring AI to your database! Integrate AI models and workflows with your database to implement custom AI applications, without moving your data. Including streaming inference, scalable model hosting, training and vector search.
Organization: superduper-io
Home Page: https://superduper.io
llm-inference,A library to communicate with ChatGPT, Claude, Copilot, Gemini, HuggingChat, and Pi
User: ugorsahin
llm-inference,A high-performance inference system for large language models, designed for production environments.
Organization: vectorch-ai
Home Page: https://docs.vectorch.com/
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.