vLLM

High-throughput LLM serving engine with PagedAttention

AI DevOps & Infrastructure

Tags

OSS

Popular

Related projects

Embedding Service

TEI-powered embedding microservice — 16ms inference, OpenAI-compatible API, one-click Railway deploy

Railway

Instant deployments with autoscaling and zero-config infrastructure

Modal

Serverless cloud for AI/ML workloads with GPU support

Replicate

Run and fine-tune open-source models with a cloud API