49 ai evaluation engineer jobs at 38 companies in Westminster, MD
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
4+ YOEBachelor's plus 6 years or master's plus 4 years in AI/ML development; 6 years programming in Python, Go, Scala, or Java; experience building scalable AI systems and leading engineering teams.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, Google Cloud, Azure, C++, C#, Golang
2mo
Save
Mark Applied
Hide
AI Engineer - SME
Annapolis Junction, Maryland, United States
$206k-$221k/yr OnsiteFull Time
Design, build, integrate, or evaluate AI/ML-enabled applications; experience with LLMs, agentic systems, and AI frameworks; strong communication.
LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, Haystack, LlamaIndex, Python, JavaScript, TypeScript
1w
Save
Mark Applied
Hide
Applied AI Lead - Evaluation & Measurement
Frederick or Beavercreek or Fort Walton Beach or Washington
OnsiteFull Time
5+ YOERequires 5+ years in data, ML, or test engineering, including 3+ years in evaluation or model validation and recent hands-on LLM evaluation. Requires a bachelor's degree or equivalent, U.S. citizenship, and active or obtainable DOD clearance.
pipelines, dashboards, large-language-model, telemetry
2mo
Save
Mark Applied
Hide
SME AI Engineer
Columbia, Maryland, United States
$225k-$247k/yr OnsiteFull Time
15+ YOEMaster's (15+ yrs) or Bachelor's (18+ yrs) and extensive experience designing, building, integrating, or evaluating AI/ML applications; familiarity with LLMs, RAG, prompt engineering, agentic frameworks (e.g., LangChain); TS/SCI w/CI Poly required.
LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, Haystack, Llamalndex, Python, JavaScript, TypeScript
2mo
Save
Mark Applied
Hide
AI Assurance Engineer
Washington, District of Columbia, United States
$142k-$236k/yr OnsiteFull Time
7+ YOE7+ years in AI testing/model validation; experience with AI evaluation frameworks, responsible AI, adversarial testing, continuous validation, and Microsoft Azure ecosystem (Azure OpenAI, Azure Synapse, Databricks). TS/SCI required.
Microsoft Azure, Azure OpenAI, Azure Synapse, Databricks, Azure Security Center
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's degree plus 6 years or master's degree plus 4 years in AI/ML; 6 years programming with Python, Go, Scala, or Java; cloud AI deployment and team leadership preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, AWS, Google Cloud, Azure, C++, C#, Golang
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or Cambridge or San Jose or New York City
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's plus 6+ years (or Master's plus 4+ years) developing AI/ML systems; strong programming (Python, Go, Scala, Java); cloud AI deployment experience; leadership and research application skills.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
3mo
Save
Mark Applied
Hide
Chief Engineer - AI Test & Evaluation (T&E)
Cambridge or Arlington
$120k-$250k/yr OnsiteFull Time
10+ YOE5+ MgmtUS citizenship; security clearance; 10+ years in AI/ML R&D and T&E; master/PhD or equivalent; proven leadership; DoD/IC program experience.
AI, Machine Learning, Computer Vision, NLP, Large Language Models, AI Agents, Data Engineering, UI Systems
3mo
Save
Mark Applied
Hide
Lead AI Engineer — Productivity Systems
Durham or Miami or Palo Alto or Washington
$12k-$18k/mo HybridFull Time
Proven experience shipping LLM-powered systems in production; strong prompt/context engineering, evaluation and guardrail development, and software engineering skills (Python/TypeScript/Clojure).
Python, TypeScript, Clojure, n8n, Zapier, AWS, Claude Code, Cursor, Copilot, Slack, Google Workspace, Jira, Terraform, RAG, MCP
1w
Save
Mark Applied
Hide
AI Engineer
Washington, District of Columbia, United States
$99k-$225k/yr HybridFull Time
3+ YOE3+ years building production AI/ML systems; bachelor's degree; experience with RAG, LLMs, cloud-native AWS architectures, APIs, observability, evaluation, deployment, and AI security; ability to obtain TS/SCI clearance.
RAG, LangChain, LlamaIndex, Pinecone, Weaviate, FAISS, OpenSearch Vector Search, FastAPI, Flask, Node.js, AWS, Amazon Bedrock, Amazon SageMaker, Amazon EKS, Amazon ECS, Amazon S3, AWS IAM, AWS Lambda, Amazon CloudWatch, AWS Secrets Manager, LangGraph, OpenTelemetry, Datadog, Redis, vLLM, Ray Serve, Triton Inference Server, Rebuff, Guardrails.ai, Databricks, Snowflake, OpenSearch, PostgreSQL, Microsoft SharePoint, Confluence, Amazon SQS, Kafka, RabbitMQ, Amazon EventBridge, Model Context Protocol (MCP)
3mo
Save
Mark Applied
Hide
AI Engineer
Columbia, Maryland, United States
$158k-$284k/yr OnsiteFull Time
18+ YOEExperience designing or evaluating AI/ML applications; strong communication; TS SCI w/Polygraph; 18+ years with BS or 15+ years with MS; familiarity with LLMs, agentic workflows, orchestration frameworks.
LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, Haystack, LlamaIndex, Python, JavaScript
2w
Save
Mark Applied
Hide
AI Solutions Engineer
Reston, Virginia, United States
$123k-$179k/yr HybridFull Time
7+ YOEBachelor's degree in a relevant field and 7+ years in AI/ML engineering or data science. Requires Python, ML frameworks, model validation, AI testing, LLM evaluation, and technical documentation experience.
Python, scikit-learn, PyTorch, TensorFlow, Hugging Face, Databricks, Azure Machine Learning, MLflow, Microsoft Copilot Studio, AutoGen, LangChain, Azure, DevOps, Git, CI/CD, RAG
1d
Save
Mark Applied
Hide
AI Engineer
Washington, D.C. or Atlanta or San Francisco
$223k-$259k/yr RemoteFull Time
5+ YOE5+ years of software engineering experience, including production LLM systems. Requires agentic systems expertise, strong Python, developer tooling, evaluation methods, systems fundamentals, ownership, and communication skills.
Python, Pydantic, MCP, OpenTelemetry, vLLM, Rust, Swift, Kotlin, Xcode, iOS simulators, CLIs, CI/CD
3d
Save
Mark Applied
Hide
Senior AI Platform Engineer, Microsoft 365 & Copilot (3 year LTE)
Seattle or Washington
$157k/yr OnsiteFull Time, Temporary
10+ YOE10+ years in AI, software, cloud, platform, Microsoft 365, or Power Platform engineering; enterprise AI experience; Microsoft cloud expertise; secure data access, governance, evaluation, and operational experience.
Microsoft 365, Microsoft Copilot, Copilot Studio, Power Platform, SharePoint, Microsoft Teams, OneDrive, Exchange, Viva, Anthropic Claude, OpenAI ChatGPT, Power Apps, Power Automate, Dataverse, Azure AI, Azure OpenAI, Microsoft Graph, Entra ID, PowerShell, Semantic Kernel, Azure AI Search, Microsoft Excel, APIs
2mo
Save
Mark Applied
Hide
AI Engineer – SME
Columbia, Maryland, United States
OnsiteFull Time
15+ YOERequires active TS/SCI with polygraph and U.S. citizenship; 15+ years experience (with a relevant advanced degree option); experience building/evaluating LLM/RAG/agentic systems and familiarity with modern AI frameworks.
LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, Haystack, LlamaIndex, Python, JavaScript/TypeScript
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
3w
Save
Mark Applied
Hide
AI Assurance Engineer
Laurel, Maryland, United States
$100k-$245k/yr OnsiteFull Time
Bachelor's in relevant technical field, Python and AI software development experience, familiarity with generative AI and test & evaluation, strong communication, and ability to obtain Interim Secret and Top Secret clearances (US citizenship).
Python
3w
Save
Mark Applied
Hide
Principal AI Engineer
Reston, Virginia, United States
$179k-$198k/yr HybridFull Time
8+ YOE8+ years engineering experience; bachelor's degree or equivalent; production experience with LLM APIs, agent frameworks, Python, cloud platforms, evaluation, security, and governance; hybrid work in Reston required.
Anthropic Claude, Claude Code, Snowflake, Omni, Model Context Protocol (MCP), Microsoft Azure, Microsoft Dynamics, Salesforce, Python, TypeScript, JavaScript, SQL, Copilot Studio, Azure AI Foundry, Power Platform
2mo
Save
Mark Applied
Hide
Agentic AI Engineer
Arlington or United States
$96k-$182k/yr HybridFull Time
U.S. citizenship and active/obtainable TS/SCI clearance required; experience building agentic/autonomous AI systems and production-quality Python systems; research evaluation and technical writing skills.
Python
2w
Save
Mark Applied
Hide
AI Platform Engineer
Arlington, Virginia, United States
$125k-$145k/yr OnsiteFull Time
3+ YOERequires 3+ years building software systems, recent LLM or agentic systems experience, Python, shell, CI, Linux, agent architecture, AI evaluation, production workloads, enterprise identity, and strong communication.
Python, MCP, CI, Linux, OAuth2, OIDC, SAML, DPO, QLoRA, GRPO, CUI, ITAR, NIST 800-171, FedRAMP, GovCloud