cgranger-sorenson Goto Github PK

backend

Common source, scripts and utilities for creating Triton backends.

checksum_repository_agent

The Triton repository agent that verifies model checksums.

common

Common source, scripts and utilities shared across all Triton repositories.

core

The core library and APIs implementing the Triton Inference Server.

dali_backend

The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.

fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

fastertransformer_backend

fil_backend

FIL backend for the Triton Inference Server

hugectr_backend

identity_backend

Example Triton backend that demonstrates most of the Triton Backend API.

local_cache

Implementation of a local in-memory cache for Triton Inference Server's TRITONCACHE API

nemo

NeMo: a toolkit for conversational AI

onnxruntime_backend

The Triton backend for the ONNX Runtime.

openvino_backend

OpenVINO backend for Triton.

paddlepaddle_backend

pyctcdecode

A fast and lightweight python-based CTC beam search decoder for speech recognition.

python_backend

Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.

pytorch_backend

The Triton backend for the PyTorch TorchScript models.

redis_cache

TRITONCACHE implementation of a Redis cache

repeat_backend

An example Triton backend that demonstrates sending zero, one, or multiple responses for each request.

server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

square_backend

Simple Triton backend used for testing.

stateful_backend

Triton backend for managing the model state tensors automatically in sequence batcher

tensorflow_backend

The Triton backend for TensorFlow.

tensorrt_backend

The Triton backend for TensorRT.

tensorrtllm_backend

The Triton TensorRT-LLM Backend

third_party

Third-party source packages that are modified for use in Triton.

vllm_backend