16 inference optimization engineer jobs at 6 companies in Arlington, VA
1w
Save
Mark Applied
Hide
Machine Learning Performance Engineer - Offboard Training & Inference
Sunnyvale or Washington, D.C. or San Diego or Fort Walton Beach or Ann Arbor or London or Stuttgart or Munich or Stockholm or Bangalore or Seoul or Tokyo
$215k-$285k/yr OnsiteFull Time
ML performance engineering experience with distributed training, batch inference, GPU or accelerator optimization, Python, and C++ or another systems language; strong debugging and analytical skills required.
FSDP, DeepSpeed, Megatron, NCCL, NVIDIA Triton Inference Server, TensorRT, ONNX Runtime, Ray, Python, C++, CUDA, Triton, CUTLASS, Nsight Systems, Nsight Compute, PyTorch Profiler, perf, Kubernetes, Slurm, ROS, OpenCV
2mo
Save
Mark Applied
Hide
Sr. Lead AI Engineer (Inference Optimization, FM hosting, AI Platform)
San Jose or San Francisco or New York City or Cambridge or McLean
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's plus 6 years or master's plus 4 years developing AI/ML technologies, and 6 years programming with Python, Go, Scala, or Java. Cloud AI deployment and team leadership are preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, NeMo Guardrails, PyTorch, Python, Go, Scala, Java, AWS, Google Cloud, Azure, C++, C#, Golang
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (FM Hosting, LLM Inference)
New York or McLean or Cambridge or San Jose
$251k-$286k/yr OnsiteFull Time
6+ YOEBachelor's in CS/AI/EE/CE or related with 6+ years (or Master's with 4+ years); 6+ years programming with Python, Go, Scala, or Java; experience deploying scalable AI systems, LLM inference, similarity search, and optimization of training/inference.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
2mo
Save
Mark Applied
Hide
Sr. Lead AI Engineer (Inference Optimization, FM hosting, AI Platform)
San Jose or San Francisco or New York City or Cambridge or McLean
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's degree plus 6 years or master's degree plus 4 years developing AI/ML technologies; 6 years programming with Python, Go, Scala, or Java; cloud AI deployment experience preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, Google Cloud, Azure, C++, C#, Golang
2w
Save
Mark Applied
Hide
Engineering Manager, Deep Learning Inference
Santa Clara or Washington or Texas or New York or Washington or Massachusetts
$184k-$357k/yr RemoteFull Time
6+ YOE3+ MgmtRequires MS, PhD, or equivalent experience; 6+ years software development; 3+ years technical leadership or engineering management; C/C++, GPU programming, performance optimization, and production deep learning deployment.
vLLM, SGLang, FlashInfer, CUDA, Triton, CUTLASS, NIXL, NCCL, NVSHMEM, C/C++, Python, PyTorch, TensorRT-LLM, Agile
2mo
Save
Mark Applied
Hide
Lead AI Engineer (Vision model customization, VLM)
New York or Cambridge or McLean or San Jose
$197k-$246k/yr OnsiteFull Time
4+ YOEBachelor's in CS/AI/EE/CE + 4 years AI/ML experience (or Master's + 2 years). 4+ years programming with Python/Go/Scala/Java. Experience with LLM inference, similarity search/VectorDBs, cloud deployments, PyTorch, and optimizing training/inference.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure, LLM
1mo
Save
Mark Applied
Hide
AI Implementation Engineer - JobID-0245
Arlington or Mount Laurel
$114k-$231k/yr HybridFull Time
5+ YOEDesign and deliver production-grade on‑prem AI/agent systems, integrate LLMs and tools, build RAG pipelines, optimize local inference, and obtain/maintain Secret clearance.
Python, NumPy, Pandas, vLLM, SGLang, Triton Inference Server, TensorRT-LLM, PyTorch, TensorFlow, LangChain, LangGraph, Semantic Kernel, AutoGen, Linux, Model Context Protocol (MCP), LLM, Retrieval-Augmented Generation (RAG)
4w
Save
Mark Applied
Hide
Engineering Manager, Deep Learning Inference
Santa Clara or Washington or Georgia or Illinois or California or Massachusetts
$224k-$431k/yr HybridFull Time
6+ YOE3+ MgmtMS/PhD or equivalent,6+ years software development with 3+ years technical leadership,strong C/C++ and Python,experience with GPU programming (CUDA,Triton,CUTLASS) and deploying/optimizing deep learning models,Agile experience.
SGLang, vLLM, FlashInfer, C/C++, Python, CUDA, Triton, CUTLASS, NIXL, NCCL, NVSHMEM, PyTorch, TensorRT-LLM
2mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (GenAI Platform, Agentic Infrastructure)
New York or San Francisco or McLean or Cambridge or San Jose or Plano
$209k-$286k/yr OnsiteFull Time
4+ YOEBachelor's in CS/AI/EE/CE +6 years or Master's +4 years; 6+ years programming with Python/Go/Scala/Java; experience deploying scalable AI on cloud; LLM, inference, similarity search, VectorDBs, guardrails, model evaluation, and optimization experience; leadership and research literacy.
Python, Go, Scala, Java, C++, C#, Golang, AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, AWS, Google Cloud, Azure
1mo
Save
Mark Applied
Hide
Lead AI Engineer
McLean or San Jose or Richmond or New York City
$197k-$225k/yr OnsiteFull Time
4+ YOEBachelor's in CS/AI/EE/CE +4 yrs (or Master's +2 yrs); 4+ yrs programming with Python, Go, Scala, or Java; experience deploying scalable AI; familiarity with LLM inference, vector DBs, guardrails, and model optimization.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer, Gen AI Platform
New York City or McLean or San Jose
$251k-$286k/yr OnsiteFull Time
6+ YOEDegree in CS/AI/EE or related plus 6+ years AI/ML engineering experience (or MS+4 yrs). Strong Python/Go/Scala/Java skills, cloud AI deployment experience, LLM/Inference and optimization expertise, leadership and communication skills.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (GenAI Platform Services)
San Jose or McLean or Cambridge or New York City
$251k-$286k/yr OnsiteFull Time
4+ YOEBachelor's in CS/AI/EE/CE plus 6+ years, or Master's plus 4+ years; 6+ years programming with Python, Go, Scala or Java; experience deploying scalable AI systems, LLM inference, vector databases, and optimization techniques; strong communication and leadership.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure, LLM
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer(MLX, Agentic AI, Gen AI platform Services)
San Jose or San Francisco or McLean or Cambridge or New York City
$230k-$286k/yr OnsiteFull Time
6+ YOE6+ years developing AI/ML systems (or 4+ with a master\u0002s), strong software and math foundations, experience with LLM inference, similarity search, and optimization, Python/Go/Scala/Java proficiency, cloud deployment experience.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
2mo
Save
Mark Applied
Hide
Lead AI Engineer (AI Foundations, LLM Core and Agentic AI)
Cambridge or McLean or New York or San Jose or United States
$197k-$246k/yr OnsiteFull Time
2+ YOEBachelor's in CS/AI/EE/CE (4+ yrs) or Master's (2+ yrs). 4+ yrs programming in Python, Go, Scala, or Java. Experience with LLMs, model training/inference, vector search, cloud deployments, and AI system optimization.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
1mo
Save
Mark Applied
Hide
Sr. GenAI Specialist PSA , Solutions Architecture
Herndon, Virginia, United States
$154k-$208k/yr OnsiteFull Time
7+ YOE7+ years designing and operating distributed applications; 5+ years customer-facing experience building production GenAI/ML systems; hands-on LLM, RAG, agentic workflows, prompt engineering, inference optimization; experience with AWS AI/ML services.
Amazon Bedrock, SageMaker, AgentCore, Bedrock Agents, Strands Agents SDK, LangChain, OpenSearch Serverless, pgvector, Pinecone
6d
Save
Mark Applied
Hide
Postdoctoral Fellow (Civil and Systems Engineering)
Baltimore, Maryland, United States
$63k-$100k/yr OnsiteFull Time
Doctoral degree in a related field. Research experience in natural hazards, AI, sensing, optimization, causal inference, generative models, infrastructure systems, or related disciplines is advantageous.
AI, LLMs
Explore Jobs
Expand Your Job Search
Top Cities Near Arlington for Inference Optimization Engineer
- Inference Optimization Engineer jobs in Washington, DC
- Inference Optimization Engineer jobs in Baltimore, MD
- Inference Optimization Engineer jobs in Alexandria, VA
- Inference Optimization Engineer jobs in Frederick, MD
- Inference Optimization Engineer jobs in Gaithersburg, MD
- Inference Optimization Engineer jobs in Rockville, MD
- Inference Optimization Engineer jobs in Bowie, MD
- Inference Optimization Engineer jobs in Leesburg, VA
- Inference Optimization Engineer jobs in Manassas, VA
- Inference Optimization Engineer jobs in Annapolis, MD
- Inference Optimization Engineer jobs in College Park, MD
- Inference Optimization Engineer jobs in Fredericksburg, VA
- Inference Optimization Engineer jobs in Laurel, MD
- Inference Optimization Engineer jobs in Herndon, VA
- Inference Optimization Engineer jobs in Greenbelt, MD
- Inference Optimization Engineer jobs in Fairfax, VA
- Inference Optimization Engineer jobs in Westminster, MD
- Inference Optimization Engineer jobs in Hyattsville, MD
- Inference Optimization Engineer jobs in Takoma Park, MD
- Inference Optimization Engineer jobs in Vienna, VA