64 ai evaluation engineer jobs at 45 companies in Manassas, VA
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
4+ YOEBachelor's plus 6 years or master's plus 4 years in AI/ML development; 6 years programming in Python, Go, Scala, or Java; experience building scalable AI systems and leading engineering teams.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, Google Cloud, Azure, C++, C#, Golang
2mo
Save
Mark Applied
Hide
AI Engineer - SME
Annapolis Junction, Maryland, United States
$206k-$221k/yr OnsiteFull Time
Design, build, integrate, or evaluate AI/ML-enabled applications; experience with LLMs, agentic systems, and AI frameworks; strong communication.
LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, Haystack, LlamaIndex, Python, JavaScript, TypeScript
1w
Save
Mark Applied
Hide
Applied AI Lead - Evaluation & Measurement
Frederick or Beavercreek or Fort Walton Beach or Washington
OnsiteFull Time
5+ YOERequires 5+ years in data, ML, or test engineering, including 3+ years in evaluation or model validation and recent hands-on LLM evaluation. Requires a bachelor's degree or equivalent, U.S. citizenship, and active or obtainable DOD clearance.
pipelines, dashboards, large-language-model, telemetry
2mo
Save
Mark Applied
Hide
SME AI Engineer
Columbia, Maryland, United States
$225k-$247k/yr OnsiteFull Time
15+ YOEMaster's (15+ yrs) or Bachelor's (18+ yrs) and extensive experience designing, building, integrating, or evaluating AI/ML applications; familiarity with LLMs, RAG, prompt engineering, agentic frameworks (e.g., LangChain); TS/SCI w/CI Poly required.
LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, Haystack, Llamalndex, Python, JavaScript, TypeScript
2mo
Save
Mark Applied
Hide
AI Assurance Engineer
Washington, District of Columbia, United States
$142k-$236k/yr OnsiteFull Time
7+ YOE7+ years in AI testing/model validation; experience with AI evaluation frameworks, responsible AI, adversarial testing, continuous validation, and Microsoft Azure ecosystem (Azure OpenAI, Azure Synapse, Databricks). TS/SCI required.
Microsoft Azure, Azure OpenAI, Azure Synapse, Databricks, Azure Security Center
2mo
Save
Mark Applied
Hide
Staff AI/ML Engineer
Chantilly or Manassas
HybridFull Time
5+ YOEBachelor's in a technical field, 5+ years AI/ML or related engineering experience, strong Python and MLOps experience, cloud (Azure/AWS/GCP), Spark/Databricks, AI testing/evaluation, and customer-facing communication skills. Secret clearance required or attainable.
Python, Jupyter Notebook, Apache Spark, Databricks, TensorFlow, PyTorch, Hugging Face, LangChain, Semantic Kernel, CrewAI, AutoGen, Kubernetes, Docker, Microsoft Azure AI Services, AWS SageMaker, Google Vertex AI, Vector databases, MLflow, GitLab, GitHub
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's degree plus 6 years or master's degree plus 4 years in AI/ML; 6 years programming with Python, Go, Scala, or Java; cloud AI deployment and team leadership preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, AWS, Google Cloud, Azure, C++, C#, Golang
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or Cambridge or San Jose or New York City
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's plus 6+ years (or Master's plus 4+ years) developing AI/ML systems; strong programming (Python, Go, Scala, Java); cloud AI deployment experience; leadership and research application skills.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
1w
Save
Mark Applied
Hide
Senior Machine Learning Engineer, AI Evaluation
Alexandria, Virginia, United States
$100k-$130k/yr HybridFull Time
7+ YOEBachelor's degree or equivalent experience and 7+ years in ML/LLM engineering, applied AI, data science, or research infrastructure. Requires Python, production systems, AI evaluation, cloud infrastructure, and reproducibility expertise.
Python, BigQuery, Vertex AI, IAM, Looker, Looker Studio, Google Cloud Platform
3mo
Save
Mark Applied
Hide
Lead AI Engineer — Productivity Systems
Durham or Miami or Palo Alto or Washington
$12k-$18k/mo HybridFull Time
Proven experience shipping LLM-powered systems in production; strong prompt/context engineering, evaluation and guardrail development, and software engineering skills (Python/TypeScript/Clojure).
Python, TypeScript, Clojure, n8n, Zapier, AWS, Claude Code, Cursor, Copilot, Slack, Google Workspace, Jira, Terraform, RAG, MCP
3mo
Save
Mark Applied
Hide
Chief Engineer - AI Test & Evaluation (T&E)
Cambridge or Arlington
$120k-$250k/yr OnsiteFull Time
10+ YOE5+ MgmtUS citizenship; security clearance; 10+ years in AI/ML R&D and T&E; master/PhD or equivalent; proven leadership; DoD/IC program experience.
AI, Machine Learning, Computer Vision, NLP, Large Language Models, AI Agents, Data Engineering, UI Systems
1w
Save
Mark Applied
Hide
AI Engineer
Washington, District of Columbia, United States
$99k-$225k/yr HybridFull Time
3+ YOE3+ years building production AI/ML systems; bachelor's degree; experience with RAG, LLMs, cloud-native AWS architectures, APIs, observability, evaluation, deployment, and AI security; ability to obtain TS/SCI clearance.
RAG, LangChain, LlamaIndex, Pinecone, Weaviate, FAISS, OpenSearch Vector Search, FastAPI, Flask, Node.js, AWS, Amazon Bedrock, Amazon SageMaker, Amazon EKS, Amazon ECS, Amazon S3, AWS IAM, AWS Lambda, Amazon CloudWatch, AWS Secrets Manager, LangGraph, OpenTelemetry, Datadog, Redis, vLLM, Ray Serve, Triton Inference Server, Rebuff, Guardrails.ai, Databricks, Snowflake, OpenSearch, PostgreSQL, Microsoft SharePoint, Confluence, Amazon SQS, Kafka, RabbitMQ, Amazon EventBridge, Model Context Protocol (MCP)
2mo
Save
Mark Applied
Hide
AI/ML Test Engineer
Springfield, Virginia, United States
OnsiteFull Time
4+ YOEU.S. citizens with active TS/SCI and CI polygraph; 4+ years supporting AI/ML testing, integration, or operational evaluation; familiarity with automated evaluation pipelines, data validation, drift detection, and troubleshooting; bachelor's in IT/STEM or equivalent experience.
3mo
Save
Mark Applied
Hide
AI Engineer
Columbia, Maryland, United States
$158k-$284k/yr OnsiteFull Time
18+ YOEExperience designing or evaluating AI/ML applications; strong communication; TS SCI w/Polygraph; 18+ years with BS or 15+ years with MS; familiarity with LLMs, agentic workflows, orchestration frameworks.
LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, Haystack, LlamaIndex, Python, JavaScript
2w
Save
Mark Applied
Hide
AI Solutions Engineer
Reston, Virginia, United States
$123k-$179k/yr HybridFull Time
7+ YOEBachelor's degree in a relevant field and 7+ years in AI/ML engineering or data science. Requires Python, ML frameworks, model validation, AI testing, LLM evaluation, and technical documentation experience.
Python, scikit-learn, PyTorch, TensorFlow, Hugging Face, Databricks, Azure Machine Learning, MLflow, Microsoft Copilot Studio, AutoGen, LangChain, Azure, DevOps, Git, CI/CD, RAG
1d
Save
Mark Applied
Hide
AI Engineer
Washington, D.C. or Atlanta or San Francisco
$223k-$259k/yr RemoteFull Time
5+ YOE5+ years of software engineering experience, including production LLM systems. Requires agentic systems expertise, strong Python, developer tooling, evaluation methods, systems fundamentals, ownership, and communication skills.
Python, Pydantic, MCP, OpenTelemetry, vLLM, Rust, Swift, Kotlin, Xcode, iOS simulators, CLIs, CI/CD
3d
Save
Mark Applied
Hide
Senior AI Platform Engineer, Microsoft 365 & Copilot (3 year LTE)
Seattle or Washington
$157k/yr OnsiteFull Time, Temporary
10+ YOE10+ years in AI, software, cloud, platform, Microsoft 365, or Power Platform engineering; enterprise AI experience; Microsoft cloud expertise; secure data access, governance, evaluation, and operational experience.
Microsoft 365, Microsoft Copilot, Copilot Studio, Power Platform, SharePoint, Microsoft Teams, OneDrive, Exchange, Viva, Anthropic Claude, OpenAI ChatGPT, Power Apps, Power Automate, Dataverse, Azure AI, Azure OpenAI, Microsoft Graph, Entra ID, PowerShell, Semantic Kernel, Azure AI Search, Microsoft Excel, APIs
2mo
Save
Mark Applied
Hide
Staff II/ Staff III AI/ML Engineer
Chantilly or Manassas
HybridFull Time
5+ YOE5+ years AI/ML experience, BS in CS or related field, hands-on MLOps and AI evaluation, cloud and distributed processing experience, strong Python skills, customer-facing communication, ability to obtain Secret clearance.
Python, Jupyter Notebook, Apache Spark, Databricks, TensorFlow, PyTorch, Hugging Face, LangChain, Semantic Kernel, CrewAI, AutoGen, Kubernetes, Docker, Azure AI Services, AWS SageMaker, Google Vertex AI, Vector databases, MLflow, GitLab, GitHub
2mo
Save
Mark Applied
Hide
AI Engineer – SME
Columbia, Maryland, United States
OnsiteFull Time
15+ YOERequires active TS/SCI with polygraph and U.S. citizenship; 15+ years experience (with a relevant advanced degree option); experience building/evaluating LLM/RAG/agentic systems and familiarity with modern AI frameworks.
LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, Haystack, LlamaIndex, Python, JavaScript/TypeScript
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)