39 llm serving engineer jobs at 36 companies in United States
1mo
Save
Mark Applied
Hide
Staff ML/LLM Ops Engineer
Seattle, Washington, United States
$213k-$272k/yr OnsiteFull Time
8+ YOE8+ years engineering experience in ML infrastructure/MLOps; hands-on LLM/VLM ops; model CI/CD, serving, monitoring, guardrails; strong API and system design; BS/MS in CS/Engineering or equivalent.
Kubernetes, Argo, NVIDIA Jetson, LangGraph, MCP, vector databases
1mo
Save
Mark Applied
Hide
Staff ML/LLM Ops Engineer
Seattle, Washington, United States
$213k-$272k/yr OnsiteFull Time
8+ YOE8+ years in ML-infrastructure/MLOps building and operating model deployment, serving, CI/CD, monitoring, and LLM/VLM ops; strong API design and technical leadership; BS/MS in CS/Engineering or equivalent experience.
Kubernetes, Argo, LangGraph, MCP, NVIDIA Jetson, vector databases
6d
Save
Mark Applied
Hide
Senior Performance Co-Design Engineer, LLM Serving
Sunnyvale, California, United States
$174k-$252k/yr OnsiteFull Time
5+ YOEBachelor's in CS/EE/CE or equivalent,5+ years in performance modeling/engineering or architecture,proficiency with C++ or Python,experience with ML serving and hardware/software co-design preferred.
C++, Python, TPU, Vertex AI
2w
Save
Mark Applied
Hide
Inference Infrastructure Engineer, Serving
Palo Alto, California, United States
$275k-$475k/yr OnsiteFull Time
3+ YOE3+ years building low-latency, high-throughput inference serving systems; knowledge of quantization, batching, speculative decoding, KV cache; experience with vLLM/TensorRT-LLM/Triton/SGLang; multi-GPU model parallelism; C++/CUDA/Python; autoscaling and GPU cost optimization.
vLLM, TensorRT-LLM, Triton, SGLang, C++, CUDA, Python
3w
Save
Mark Applied
Hide
Research Engineer / Scientist - Storage for LLM
Seattle, Washington, United States
OnsiteFull Time
PhD in a technical field; deep expertise in distributed storage and KV caching for LLMs, transformer internals, low-latency serving, GPU environments, and systems programming in C++/Rust/Go/CUDA.
CUDA, Triton Inference Server, TensorRT, RPC, gRPC, NVIDIA GPUs, C++, Rust, Go, vLLM, SGLang, FasterTransformer, DeepSpeed, Hugging Face Text Generation Inference, NCCL, GDR, GDS, InfiniBand, RDMA, HBM, NUMA, NVLink
2mo
Save
Mark Applied
Hide
Machine Learning Engineer, Inference & Serving (Speech LLM) - San Francisco
San Francisco, California, United States
$180k-$270k/yr HybridFull Time
Experience building and deploying high-throughput, ultra-low-latency inference for LLMs or speech models; optimize latency/throughput; manage KV cache; understand GPU memory hierarchies; collaborate across ML and backend teams.
vLLM, TensorRT-LLM, SGLang, NVIDIA Triton Inference Server, WebSockets, WebRTC, CUDA, PTQ, FP8, INT8, AWQ, GPTQ, Tensor Parallelism, Kubernetes
1mo
Save
Mark Applied
Hide
AI Engineer (Applied LLM Systems)
New York City, New York, United States
$165k-$200k/yr OnsiteFull Time
Experience building production LLM/ML features end-to-end, hands-on with agentic orchestration, self-hosted model serving, strong Python and evaluation design skills.
vLLM, TGI, Python
3w
Save
Mark Applied
Hide
Principal ML Engineer
New York, New York, United States
$200k-$250k/yr HybridFull Time
8+ YOE8+ years software engineering experience with 4+ years building ML/LLM infrastructure, hands-on PyTorch and LLM stack, production model serving, strong Python and cloud skills.
PyTorch, LoRA, SFT, vLLM, TensorRT-LLM, Python, CI/CD
1mo
Save
Mark Applied
Hide
Infrastructure Engineer
Redwood City, California, United States
HybridFull Time
8+ YOE8+ years in infrastructure/DevOps/SRE; strong cloud expertise; experience with CI/CD, PostgreSQL, Redis, APM, model serving, vector databases, GPU optimization, and LLM deployment.
PostgreSQL, Redis, APM, CI/CD, vector databases, model serving frameworks, LLM
2mo
Save
Mark Applied
Hide
AI Platform Engineer, Training and Inference
San Francisco, California, United States
HybridFull Time
ML platform or MLOps engineer with production Ray experience; LLM serving, distributed training, Python and PyTorch; MLflow/Flyte; Bachelor's degree in CS/Engineering.
Ray Train, Ray Serve, Ray Core, Ray Data, vLLM, SGLang, NVIDIA Triton, TorchTrainer, DDP, NCCL, PPO, RLlib, Flyte, MLflow, Qdrant, Pgvector, PyTorch, Python
6d
Save
Mark Applied
Hide
Distinguished AI Engineer
Jersey City, New Jersey, United States
$140k-$210k/yr OnsiteFull Time
10+ YOE10+ years in AI/ML, distributed systems, enterprise architecture or platform engineering; deep LLM, RAG, embeddings, model-serving and AWS experience; Terraform/IaC and DevOps pipeline expertise; ability to influence senior stakeholders.
AWS, Terraform, IaC, DevOps, LLMs, RAG
2mo
Save
Mark Applied
Hide
Senior AI/ML Engineer
San Francisco, California, United States
OnsiteFull Time
4+ YOE4+ years applied ML engineering in production, experience with LLMs/fine-tuning/RAG or large-scale recommender systems, strong Python and PyTorch or JAX skills, distributed training/GPU/inference serving experience, and mentoring ability.
Python, PyTorch, JAX
1mo
Save
Mark Applied
Hide
AI Field Engineer - AI Natives
San Mateo, California, United States
FieldFull Time
5+ YOE5+ years in customer-facing technical engineering roles, strong Python and Kubernetes skills, experience with LLM inference, model serving and fine-tuning, cloud GPU deployment across major clouds, and exceptional communication.
Python, Kubernetes, vLLM, SGLang, TensorRT-LLM, AWS, Microsoft Azure, GCP, Azure AI Foundry, AWS Bedrock, SageMaker, GCP Vertex
2w
Save
Mark Applied
Hide
Senior MLOps Engineer
Palo Alto, California, United States
OnsiteFull Time
5+ YOE5+ years MLOps or ML infrastructure experience, BS in CS or related, strong Python, Kubernetes, Docker, Terraform, GPU cluster and LLM serving experience, proficiency with orchestration and observability tools.
vLLM, TRT, SGLang, TGI, Kubernetes, Docker, Terraform, Python, Airflow, Kubeflow, Ray, MLflow, Weights & Biases, AWS, GCP, Azure, Spark, AWQ, GPTQ, FP8/GGUF
1mo
Save
Mark Applied
Hide
AI Platform Engineer
Boston or Springfield
$134k-$176k/yr HybridFull Time
2+ YOE2+ years platform/infrastructure/SRE experience; cloud-native experience (AWS, GCP, or Azure); delivered platform features to production; experience with IaC and GitOps (Terraform or Pulumi, ArgoCD); familiarity with Kubernetes and LLM serving frameworks.
AWS, GCP, Azure, Terraform, Pulumi, ArgoCD, Kubernetes, vLLM, Triton, Ray Serve, Degreed
1d
Save
Mark Applied
Hide
Principal AI Platform Engineer
Atlanta, Georgia, United States
$142k-$203k/yr OnsiteFull Time
Experience leading AI platform or infrastructure teams, hands-on LLM inference/serving, cloud (AWS/Azure/GCP) with Kubernetes and IaC, regulated-industry delivery, research fluency, and platform-as-product mindset.
Lang Smith, Lang Graph Platform, model gateway, vector search, PostgreSQL, pg vector, ClickHouse, S3-compatible object storage, Neo4j Enterprise, Open Telemetry, Grafana, Kubernetes, Helm, Argo CD, AWS, Azure, GCP
2w
Save
Mark Applied
Hide
Machine Learning Engineer (Staff)
San Francisco or Menlo Park
$220k-$270k/yr HybridFull Time
8+ YOE8+ years building production ML systems and infrastructure; experience with training/serving pipelines, feature pipelines, monitoring, deployment, cloud, containers, CI/CD, and model governance.
CI/CD, APIs, containers, feature stores, MLOps, LLM
1w
Save
Mark Applied
Hide
Senior Engineer, Inference Data Plane
Denver or Seattle
$139k-$174k/yr RemoteFull Time
Design and deliver high-scale, resilient data-plane services for inference; experience with LLM serving, distributed systems, and AI hardware; expert in Go or Python and gRPC; familiarity with inference frameworks and observability.
llm-d, vLLM, SGLang, TensorRT, NVIDIA Dynamo, Ray Serve, KServe, TensorRT-LLM, TGI, Modular MAX, gRPC, GoLang, Python, Kubernetes
2w
Save
Mark Applied
Hide
Senior AI Engineer
Chicago or Hong Kong or London or New York City or Singapore
$200k-$300k/yr OnsiteFull Time
5+ YOE5+ years software engineering with strong Python; production fine-tuning/distillation of open-weight models; on-prem LLM serving, GPU infrastructure and Kubernetes experience; model evaluation and tooling.
Python, Llama, Qwen, Mistral, vLLM, TGI, Triton, Kubernetes, Hugging Face
3w
Save
Mark Applied
Hide
Senior Machine Learning Engineer
United States
$235k-$250k/yr RemoteFull Time
7+ YOE7+ years engineering experience with 5+ years shipping production ML; strong Python and SQL; experience with MLflow, AWS, LLMs, production model serving, CI/CD, and feature engineering.
Python, SQL, MLflow, AWS, scikit-learn, LLMs
Explore Jobs