45 ai evaluation engineer jobs at 28 companies in Virginia
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
4+ YOEBachelor's plus 6 years or master's plus 4 years in AI/ML development; 6 years programming in Python, Go, Scala, or Java; experience building scalable AI systems and leading engineering teams.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, Google Cloud, Azure, C++, C#, Golang
2mo
Save
Mark Applied
Hide
Staff AI/ML Engineer
Chantilly or Manassas
HybridFull Time
5+ YOEBachelor's in a technical field, 5+ years AI/ML or related engineering experience, strong Python and MLOps experience, cloud (Azure/AWS/GCP), Spark/Databricks, AI testing/evaluation, and customer-facing communication skills. Secret clearance required or attainable.
Python, Jupyter Notebook, Apache Spark, Databricks, TensorFlow, PyTorch, Hugging Face, LangChain, Semantic Kernel, CrewAI, AutoGen, Kubernetes, Docker, Microsoft Azure AI Services, AWS SageMaker, Google Vertex AI, Vector databases, MLflow, GitLab, GitHub
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's degree plus 6 years or master's degree plus 4 years in AI/ML; 6 years programming with Python, Go, Scala, or Java; cloud AI deployment and team leadership preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, AWS, Google Cloud, Azure, C++, C#, Golang
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or Cambridge or San Jose or New York City
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's plus 6+ years (or Master's plus 4+ years) developing AI/ML systems; strong programming (Python, Go, Scala, Java); cloud AI deployment experience; leadership and research application skills.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
1w
Save
Mark Applied
Hide
Senior Machine Learning Engineer, AI Evaluation
Alexandria, Virginia, United States
$100k-$130k/yr HybridFull Time
7+ YOEBachelor's degree or equivalent experience and 7+ years in ML/LLM engineering, applied AI, data science, or research infrastructure. Requires Python, production systems, AI evaluation, cloud infrastructure, and reproducibility expertise.
Python, BigQuery, Vertex AI, IAM, Looker, Looker Studio, Google Cloud Platform
3mo
Save
Mark Applied
Hide
Chief Engineer - AI Test & Evaluation (T&E)
Cambridge or Arlington
$120k-$250k/yr OnsiteFull Time
10+ YOE5+ MgmtUS citizenship; security clearance; 10+ years in AI/ML R&D and T&E; master/PhD or equivalent; proven leadership; DoD/IC program experience.
AI, Machine Learning, Computer Vision, NLP, Large Language Models, AI Agents, Data Engineering, UI Systems
3w
Save
Mark Applied
Hide
AI Engineer - VA
Virginia, United States
OnsiteFull Time
5+ YOE5+ years of AI/ML engineering experience evaluating, fine-tuning, and deploying LLMs in production; RAG, embeddings, vector databases, agentic workflows, EvalOps, cloud infrastructure, and production software engineering required.
large language models (LLMs), RAG, embedding models, vector databases, AWS, GCP, EvalOps, Evals-as-a-Service
2mo
Save
Mark Applied
Hide
AI/ML Test Engineer
Springfield, Virginia, United States
OnsiteFull Time
4+ YOEU.S. citizens with active TS/SCI and CI polygraph; 4+ years supporting AI/ML testing, integration, or operational evaluation; familiarity with automated evaluation pipelines, data validation, drift detection, and troubleshooting; bachelor's in IT/STEM or equivalent experience.
2w
Save
Mark Applied
Hide
AI Solutions Engineer
Reston, Virginia, United States
$123k-$179k/yr HybridFull Time
7+ YOEBachelor's degree in a relevant field and 7+ years in AI/ML engineering or data science. Requires Python, ML frameworks, model validation, AI testing, LLM evaluation, and technical documentation experience.
Python, scikit-learn, PyTorch, TensorFlow, Hugging Face, Databricks, Azure Machine Learning, MLflow, Microsoft Copilot Studio, AutoGen, LangChain, Azure, DevOps, Git, CI/CD, RAG
2mo
Save
Mark Applied
Hide
Staff II/ Staff III AI/ML Engineer
Chantilly or Manassas
HybridFull Time
5+ YOE5+ years AI/ML experience, BS in CS or related field, hands-on MLOps and AI evaluation, cloud and distributed processing experience, strong Python skills, customer-facing communication, ability to obtain Secret clearance.
Python, Jupyter Notebook, Apache Spark, Databricks, TensorFlow, PyTorch, Hugging Face, LangChain, Semantic Kernel, CrewAI, AutoGen, Kubernetes, Docker, Azure AI Services, AWS SageMaker, Google Vertex AI, Vector databases, MLflow, GitLab, GitHub
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1mo
Save
Mark Applied
Hide
Senior AI Platform Engineer
San Francisco or Alabama or Arkansas or California or Colorado or Florida or Georgia or Illinois or Kentucky or Massachusetts or Michigan or Montana or Missouri or Nevada or New Jersey or New York or North Carolina or Oregon or Pennsylvania or Texas or Washington or Utah or Virginia or Wisconsin
$156k-$231k/yr HybridFull Time
3+ YOE3+ years building production software or ML/data systems; strong Python and systems/API design; experience with LLM retrieval/RAG, agents, evaluation, Databricks/MLflow and modern data stacks.
Python, PySpark, dbt, Databricks, Delta Lake, Unity Catalog, MLflow, Astronomer, Airflow, Claude, GitHub, Shortcut, Notion, pgvector, Pinecone, Weaviate, LangGraph, MCP
3w
Save
Mark Applied
Hide
Principal AI Engineer
Reston, Virginia, United States
$179k-$198k/yr HybridFull Time
8+ YOE8+ years engineering experience; bachelor's degree or equivalent; production experience with LLM APIs, agent frameworks, Python, cloud platforms, evaluation, security, and governance; hybrid work in Reston required.
Anthropic Claude, Claude Code, Snowflake, Omni, Model Context Protocol (MCP), Microsoft Azure, Microsoft Dynamics, Salesforce, Python, TypeScript, JavaScript, SQL, Copilot Studio, Azure AI Foundry, Power Platform
2mo
Save
Mark Applied
Hide
Agentic AI Engineer
Arlington or United States
$96k-$182k/yr HybridFull Time
U.S. citizenship and active/obtainable TS/SCI clearance required; experience building agentic/autonomous AI systems and production-quality Python systems; research evaluation and technical writing skills.
Python
2w
Save
Mark Applied
Hide
AI Platform Engineer
Arlington, Virginia, United States
$125k-$145k/yr OnsiteFull Time
3+ YOERequires 3+ years building software systems, recent LLM or agentic systems experience, Python, shell, CI, Linux, agent architecture, AI evaluation, production workloads, enterprise identity, and strong communication.
Python, MCP, CI, Linux, OAuth2, OIDC, SAML, DPO, QLoRA, GRPO, CUI, ITAR, NIST 800-171, FedRAMP, GovCloud
2w
Save
Mark Applied
Hide
Audio AI Engineer
Reston, Virginia, United States
$80k-$160k/yr RemoteFull Time
Bachelor's degree and experience building audio/text data pipelines, adapting multilingual ASR models, evaluating speech recognition, deploying production ML systems, and using Python, SQL, PyTorch, and related tools.
LoRA, QLoRA, Python, SQL, PyTorch, Hugging Face Transformers, PEFT, torchaudio, librosa, iPhone, iOS, Swift, AVFoundation
1mo
Save
Mark Applied
Hide
Senior Forward Deployed AI Engineer (GenAI, AWS)
New Jersey or New York or Pennsylvania or North Carolina or Connecticut or Massachusetts or Virginia
RemoteFull Time
Provectus: Builds and deploys custom artificial intelligence and data solutions.
8+ YOE8+ years building production software, hands-on coding (Python/TypeScript), cloud-native AWS delivery, shipped GenAI/LLM systems to production, built evaluation suites, strong stakeholder communication and domain learning ability.
Claude, Claude Code, Cowork, AWS, Python, TypeScript, Kubernetes, ECS, PyTorch, SageMaker, MLflow, Neo4j, AWS Neptune, AWS CDK, CloudFormation, Terraform, ChatGPT
1w
Save
Mark Applied
Hide
Lead Maritime Test and Evaluation Director
Arlington, Virginia, United States
$147k-$200k/yr OnsiteFull Time
8+ YOEBS in Mechanical or Ocean Engineering with 8+ years of relevant experience, or equivalent; extensive maritime test, systems integration, technical risk, and readiness assessment experience; ability to obtain Top Secret clearance and U.S. citizenship required.
Hardware-in-the-Loop (HIL), Software-in-the-Loop (SIL), AI/ML
3w
Save
Mark Applied
Hide
Manager, Conversational AI
United States or California or Florida or Georgia or Idaho or Illinois or Massachusetts or Michigan or New Jersey or New York or North Carolina or Ohio or Oregon or Pennsylvania or Texas or Virginia or Washington or Wisconsin
RemoteFull Time
Experience leading teams and building conversational AI products, including LLM-powered agents, evaluation frameworks, optimization, experimentation, and cross-functional delivery with Product, Engineering, Data, and AI teams.
AI, Machine Learning, LLM
Explore Jobs
Expand Your Job Search
Top Cities in Virginia for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Virginia Beach, VA
- AI Evaluation Engineer jobs in Norfolk, VA
- AI Evaluation Engineer jobs in Chesapeake, VA
- AI Evaluation Engineer jobs in Richmond, VA
- AI Evaluation Engineer jobs in Arlington, VA
- AI Evaluation Engineer jobs in Newport News, VA
- AI Evaluation Engineer jobs in Alexandria, VA
- AI Evaluation Engineer jobs in Hampton, VA
- AI Evaluation Engineer jobs in Roanoke, VA
- AI Evaluation Engineer jobs in Portsmouth, VA
- AI Evaluation Engineer jobs in Suffolk, VA
- AI Evaluation Engineer jobs in Lynchburg, VA
- AI Evaluation Engineer jobs in Harrisonburg, VA
- AI Evaluation Engineer jobs in Leesburg, VA
- AI Evaluation Engineer jobs in Charlottesville, VA
- AI Evaluation Engineer jobs in Blacksburg, VA
- AI Evaluation Engineer jobs in Danville, VA
- AI Evaluation Engineer jobs in Manassas, VA
- AI Evaluation Engineer jobs in Petersburg, VA
- AI Evaluation Engineer jobs in Fredericksburg, VA