51 ai evaluation engineer jobs at 27 companies in Redmond, WA
4w
Save
Mark Applied
Hide
AI Automation Engineer L1
Bellevue, Washington, United States
$100k-$180k/yr OnsiteFull Time
8+ YOE8+ years experience in enterprise platform engineering with Python and AI/ML ops; designing AI testing frameworks, performance testing, and CI/CD-integrated LLM evaluation pipelines.
Python, SQL, LLM, CI/CD
1mo
Save
Mark Applied
Hide
Agentic AI Engineer
Seattle, Washington, United States
$130k-$170k/yr HybridFull Time
3+ YOE3+ years software engineering experience, 1+ year shipping LLM/AI applications, hands-on with AWS AI services (Bedrock, Strands, AgentCore, SageMaker), strong Python/TypeScript skills, prompt engineering, evaluation, and responsible AI practices.
Strands Agents, AgentCore, Bedrock, Lambda, SageMaker, Python, TypeScript, IAM, DataDog, GitHub CI/CD
1d
Save
Mark Applied
Hide
AI Developer
Scottsdale or London or Bellevue
$135k-$165k/yr HybridFull Time
4+ YOE4+ years in AI/ML engineering; strong Python and ML framework skills; LLM, embeddings, vector database, RAG, cloud deployment, DevOps, API, evaluation, and monitoring experience required.
Python, TensorFlow, PyTorch, Scikit-learn, OpenAI, Azure OpenAI, Hugging Face, Pinecone, Weaviate, Redis, Docker, Azure, AWS, GCP, CI/CD
3d
Save
Mark Applied
Hide
Senior AI Platform Engineer, Microsoft 365 & Copilot (3 year LTE)
Seattle or Washington
$157k/yr OnsiteFull Time, Temporary
10+ YOE10+ years in AI, software, cloud, platform, Microsoft 365, or Power Platform engineering; enterprise AI experience; Microsoft cloud expertise; secure data access, governance, evaluation, and operational experience.
Microsoft 365, Microsoft Copilot, Copilot Studio, Power Platform, SharePoint, Microsoft Teams, OneDrive, Exchange, Viva, Anthropic Claude, OpenAI ChatGPT, Power Apps, Power Automate, Dataverse, Azure AI, Azure OpenAI, Microsoft Graph, Entra ID, PowerShell, Semantic Kernel, Azure AI Search, Microsoft Excel, APIs
2w
Save
Mark Applied
Hide
Staff AI Platform Engineer
San Francisco or Los Angeles or Denver or Austin or Chicago or New York or Seattle or Toronto or Santa Barbara or San Diego
$180k-$273k/yr RemoteFull Time
7+ YOERequires 7+ years in AI/ML platform engineering, strong Python and/or TypeScript, distributed systems, APIs, evaluation, observability, and AI infrastructure experience; bachelor's degree or equivalent practical experience.
Python, TypeScript, JavaScript, React, Kubernetes, Docker, Terraform, LangChain, LangGraph, LlamaIndex, LiveKit, REST, gRPC, MCP, A2A
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1d
Save
Mark Applied
Hide
AI Engineer, GTM Claudification
San Francisco or Seattle
$320k-$405k/yr HybridFull Time
8+ YOEProduction engineering experience with Python or TypeScript, LLMs, agents, MCP, evaluations, SQL, and data-driven systems; 8+ years preferred. Bachelor's degree or equivalent experience required.
Python, TypeScript, SQL, MCP, Claude Code, Claude Agent SDK, CRM
1w
Save
Mark Applied
Hide
Applied AI Engineer
San Francisco or Los Angeles or Washington or San Diego or Seattle or London
$150k-$170k/yr HybridFull Time
2+ YOEBachelor's or master's degree in a technical field, 2+ years of software development, strong Python, production systems, AI/ML integration, model evaluation, and familiarity with APIs, distributed systems, containers, CI/CD, observability, and edge deployment.
Python, APIs, CI/CD, GPU
3w
Save
Mark Applied
Hide
Frontier Agents Engineer (Applied AI)
San Francisco or Seattle or New York City
$180k-$225k/yr OnsiteFull Time
4+ YOE4+ years in software engineering, ML, or applied AI; strong Python; experience building production AI systems with LLMs; experience with OpenAI/Claude/MCP/agent frameworks and quantitative evaluation; excellent communication.
Python, OpenAI, Claude, MCP, agent frameworks, vector databases, LLMs, Docker, Kubernetes, CI/CD, AWS, Azure, GCP
3mo
Save
Mark Applied
Hide
Senior AI/ML Engineer
Seattle, Washington, United States
$177k-$232k/yr HybridFull Time
6+ YOE6+ years building and delivering AI/ML solutions; Bachelor's in CS/ML or related; proficiency with Python, Spark, Airflow, MLflow; experience with MLOps, model evaluation, and production ML systems.
Python, Spark, Airflow, MLflow, MLOps
6d
Save
Mark Applied
Hide
AI Architect
Bangalore or Milpitas or Seattle or Princeton or Cape Town or London or Zurich or Singapore or Mexico City
RemoteFull Time
17+ YOERequires 17–21 years of technology experience with substantial quality engineering and test engineering expertise, agentic AI architecture, evaluation-driven development, client presentations, presales, and AI assurance standards knowledge.
ZenseAI.QI, ZenseAI.AssureAI, DeepSpeci, CaseGeni, Auto-PlayPilot, DataGeni, Secure-Xi, Perf-Xi, Insights360, RIA, Agentic Foundry, LLM-as-judge, RFP, RFI, EU AI Act, NIST AI RMF, ISO/IEC 42001
3d
Save
Mark Applied
Hide
Senior AI Platform Engineer, Microsoft 365 & Copilot (3 year LTE)
Seattle or Washington
$157k/yr OnsiteFull Time, Temporary
10+ YOE10+ years in AI, software, cloud, platform, Microsoft 365, or Power Platform engineering; enterprise AI and Microsoft cloud experience; secure data access, governance, evaluation, operations, documentation, and communication skills.
Microsoft 365, Microsoft 365 Copilot, Copilot Studio, Power Platform, SharePoint, Microsoft Teams, OneDrive, Exchange, Viva, Anthropic Claude, OpenAI ChatGPT, Azure AI, Azure OpenAI, Microsoft Graph, Power Apps, Power Automate, Dataverse, PowerShell, Entra ID, Semantic Kernel, Azure AI Search, APIs
3mo
Save
Mark Applied
Hide
AI Research Engineer - Social Products (Technical Leadership)
Bellevue or Menlo Park or Seattle or New York
$219k-$301k/yr HybridFull Time
5+ YOEBachelor's degree in CS/CE or related field; 5+ years inApplied AI; experience with large-scale model training and evaluating speech-based systems.
1mo
Save
Mark Applied
Hide
Staff Machine Learning Engineer - AI Cloud
San Francisco or New York or Seattle
$198k-$330k/yr OnsiteFull Time
8+ YOE8+ years software engineering experience with 5+ years focused on ML; deep expertise in LLMs and modern ML; hands-on with PyTorch and JAX; experience shipping ML systems and building evaluation/production infrastructure.
PyTorch, JAX, Slack, Microsoft 365
3w
Save
Mark Applied
Hide
Application Software Engineer, Applied AI
Bastrop or Cape Canaveral or Hawthorne or McGregor or Redmond or Brownsville or Vandenberg
OnsiteFull Time
1+ YOEDesign, evaluate, deploy, and integrate AI systems including LLMs; 1+ years full-stack experience; bachelor's in CS/engineering/math or 2+ years professional software experience; strong software engineering and AI production experience.
Python, C#.NET, Go, Scala, Java, Docker, Kubernetes, Kafka, Angular, React, PostgreSQL, SQL Server, LLM
3w
Save
Mark Applied
Hide
Technical Support Engineer - AI Lead
Redmond or Fargo or Las Colinas or Charlotte or San Francisco or New York City
$131k-$277k/yr HybridFull Time
10+ YOEBachelor's degree in computer science, IT, or related field and 10+ years of technical support, consulting, or IT experience; expertise in AI/ML, governance, evaluation, and executive stakeholder management.
AI, Responsible AI, LLM, agentic, AI/ML
1d
Save
Mark Applied
Hide
Machine Learning Engineer, Human Centered AI - Evaluations & Insights
Seattle or Washington
OnsiteFull Time
5+ YOE5+ years in ML engineering or applied research; advanced Python and deep learning expertise; experience with scalable inference, model evaluation, LLMs, NLP, MLOps, prompt engineering, RAG, and AI alignment.
Python, PyTorch, JAX, Hugging Face, SelfCheckGPT, MLflow, Weights & Biases, G-Eval, DeepEval, RAG
3w
Save
Mark Applied
Hide
Senior Software Engineer, Agentic AI
Redmond or Santa Clara
$152k-$288k/yr OnsiteFull Time
4+ YOEMS in CS or equivalent experience,4+ years building large-scale production systems,experience with agent design, AI evaluation, distributed systems; proficient in Python and C++/Rust.
Python, C++, Rust, Claude Code, Cursor, Codex, CUDA, Nsight Compute, Nsight Systems, TRT-LLM, SGLang, vLLM, Transformer Engine
3w
Save
Mark Applied
Hide
Senior Software Engineer, Agentic AI
Redmond or Santa Clara
$152k-$288k/yr OnsiteFull Time
4+ YOEMS in CS/CE or equivalent experience, 4+ years building large-scale production systems, experience with coding agents, AI evaluation, distributed systems, Python and C++/Rust proficiency.
Claude Code, Cursor, Codex, Python, C++, Rust, CUDA, Nsight Compute, Nsight Systems, TRT-LLM, SGLang, vLLM, Transformer Engine
2mo
Save
Mark Applied
Hide
Information Technology - Principal Platform Engineer, Enterprise AI Enablement
Vancouver or Seattle or Toronto or New York City or Los Angeles
OnsiteFull Time
Deep enterprise AI and platform architecture experience, LLMOps/MLOps, AI guardrails and observability, vector DBs and data orchestration, vendor evaluation, and cross-functional leadership for enterprise AI enablement.
Azure AI Studio, AWS Bedrock, Google Cloud Vertex AI, LLMOps, MLOps, RAG
Explore Jobs
Expand Your Job Search
Top Cities Near Redmond for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Seattle, WA
- AI Evaluation Engineer jobs in Tacoma, WA
- AI Evaluation Engineer jobs in Bellevue, WA
- AI Evaluation Engineer jobs in Kent, WA
- AI Evaluation Engineer jobs in Everett, WA
- AI Evaluation Engineer jobs in Renton, WA
- AI Evaluation Engineer jobs in Federal Way, WA
- AI Evaluation Engineer jobs in Kirkland, WA
- AI Evaluation Engineer jobs in Auburn, WA
- AI Evaluation Engineer jobs in Marysville, WA
- AI Evaluation Engineer jobs in Lakewood, WA
- AI Evaluation Engineer jobs in Shoreline, WA
- AI Evaluation Engineer jobs in Sammamish, WA
- AI Evaluation Engineer jobs in Burien, WA
- AI Evaluation Engineer jobs in Bothell, WA
- AI Evaluation Engineer jobs in Edmonds, WA
- AI Evaluation Engineer jobs in Puyallup, WA
- AI Evaluation Engineer jobs in Bremerton, WA
- AI Evaluation Engineer jobs in Lynnwood, WA
- AI Evaluation Engineer jobs in Issaquah, WA