14 ai evaluation engineer jobs at 12 companies in Winchester, VA
1w
Save
Mark Applied
Hide
Applied AI Lead - Evaluation & Measurement
Frederick or Beavercreek or Fort Walton Beach or Washington
OnsiteFull Time
5+ YOERequires 5+ years in data, ML, or test engineering, including 3+ years in evaluation or model validation and recent hands-on LLM evaluation. Requires a bachelor's degree or equivalent, U.S. citizenship, and active or obtainable DOD clearance.
pipelines, dashboards, large-language-model, telemetry
2mo
Save
Mark Applied
Hide
Staff AI/ML Engineer
Chantilly or Manassas
HybridFull Time
5+ YOEBachelor's in a technical field, 5+ years AI/ML or related engineering experience, strong Python and MLOps experience, cloud (Azure/AWS/GCP), Spark/Databricks, AI testing/evaluation, and customer-facing communication skills. Secret clearance required or attainable.
Python, Jupyter Notebook, Apache Spark, Databricks, TensorFlow, PyTorch, Hugging Face, LangChain, Semantic Kernel, CrewAI, AutoGen, Kubernetes, Docker, Microsoft Azure AI Services, AWS SageMaker, Google Vertex AI, Vector databases, MLflow, GitLab, GitHub
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's degree plus 6 years or master's degree plus 4 years in AI/ML; 6 years programming with Python, Go, Scala, or Java; cloud AI deployment and team leadership preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, AWS, Google Cloud, Azure, C++, C#, Golang
2w
Save
Mark Applied
Hide
AI Solutions Engineer
Reston, Virginia, United States
$123k-$179k/yr HybridFull Time
7+ YOEBachelor's degree in a relevant field and 7+ years in AI/ML engineering or data science. Requires Python, ML frameworks, model validation, AI testing, LLM evaluation, and technical documentation experience.
Python, scikit-learn, PyTorch, TensorFlow, Hugging Face, Databricks, Azure Machine Learning, MLflow, Microsoft Copilot Studio, AutoGen, LangChain, Azure, DevOps, Git, CI/CD, RAG
2mo
Save
Mark Applied
Hide
Staff II/ Staff III AI/ML Engineer
Chantilly or Manassas
HybridFull Time
5+ YOE5+ years AI/ML experience, BS in CS or related field, hands-on MLOps and AI evaluation, cloud and distributed processing experience, strong Python skills, customer-facing communication, ability to obtain Secret clearance.
Python, Jupyter Notebook, Apache Spark, Databricks, TensorFlow, PyTorch, Hugging Face, LangChain, Semantic Kernel, CrewAI, AutoGen, Kubernetes, Docker, Azure AI Services, AWS SageMaker, Google Vertex AI, Vector databases, MLflow, GitLab, GitHub
3w
Save
Mark Applied
Hide
Principal AI Engineer
Reston, Virginia, United States
$179k-$198k/yr HybridFull Time
8+ YOE8+ years engineering experience; bachelor's degree or equivalent; production experience with LLM APIs, agent frameworks, Python, cloud platforms, evaluation, security, and governance; hybrid work in Reston required.
Anthropic Claude, Claude Code, Snowflake, Omni, Model Context Protocol (MCP), Microsoft Azure, Microsoft Dynamics, Salesforce, Python, TypeScript, JavaScript, SQL, Copilot Studio, Azure AI Foundry, Power Platform
3mo
Save
Mark Applied
Hide
AI Architect
Chantilly, Virginia, United States
HybridFull Time
10+ YOE10+ years building and operationalizing enterprise-scale AI/ML solutions; experience with LLMs, agentic systems, MLOps, evaluation frameworks, and secure AI practices; Bachelor’s degree required.
Python, Jupyter Notebook, Apache Spark, Databricks, TensorFlow, PyTorch, Hugging Face, LangChain, Semantic Kernel, CrewAI, AutoGen, Kubernetes, Docker, Microsoft Azure AI Services, AWS SageMaker, Google Vertex AI, Vector databases, MLflow, GitLab, GitHub CI/CD pipelines
2w
Save
Mark Applied
Hide
Audio AI Engineer
Reston, Virginia, United States
$80k-$160k/yr RemoteFull Time
Bachelor's degree and experience building audio/text data pipelines, adapting multilingual ASR models, evaluating speech recognition, deploying production ML systems, and using Python, SQL, PyTorch, and related tools.
LoRA, QLoRA, Python, SQL, PyTorch, Hugging Face Transformers, PEFT, torchaudio, librosa, iPhone, iOS, Swift, AVFoundation
1w
Save
Mark Applied
Hide
Senior System Safety Engineer
Manassas, Virginia, United States
$96k-$178k/yr OnsiteFull Time
Requires MIL-STD-882 safety program experience, system-of-systems evaluation, systems engineering across the development lifecycle, partner communication, AI prompting, and ability to obtain a DoD Secret clearance with US citizenship.
MIL-STD-882, CAMEO, Magic Draw, Agile
2mo
Save
Mark Applied
Hide
Senior Applied AI Solutions Architect — Amazon Connect, Applied AI SA - AIVT
Seattle or Arlington or Atlanta or New York or Denver or Herndon or Chicago or Mountain View or San Francisco
$154k-$239k/yr OnsiteFull Time
7+ YOE7+ years in technology domains; 3+ years in design/implementation; familiarity with MCP/A2A; AI/ML concepts incl. LLMs, prompt engineering, RAG, model evaluation
Python, Node.js, AWS Lambda, API Gateway, Step Functions, Bedrock, MCP, RAG, DynamoDB, RDS, S3, OpenSearch, Kendra
2mo
Save
Mark Applied
Hide
Space-to-Ground Systems Integration Engineer
El Segundo or Chantilly or Colorado Springs
$100k-$176k/yr OnsiteFull Time
5+ YOEBachelor's in engineering/physics/computer science (or related); 5+ years Test & Evaluation and space systems integration experience (8+ for specialist); secret clearance required (active preferred or ability to obtain); experience with requirements flow-down, SIP/SEP/STIP, test planning/execution, cloud (AWS, Microsoft Azure, GCP), systems enginee...
Cameo, STK, Simulink, Python, MATLAB, AWS, Microsoft Azure, GCP, CI/CD, DevSecOps, AI/ML
2w
Save
Mark Applied
Hide
Senior Software/System Test Engineer (Remote)
Sterling, Virginia, United States
$40-$70/hr RemotePart Time, Contract
7+ YOERequires 7+ years in software testing, ISO/IEC 25010:2023 implementation, software or AI systems evaluation, test planning, execution, evidence capture, defect documentation, and strong reporting skills.
ISO/IEC 25010:2023, JIRA, WCAG
2w
Save
Mark Applied
Hide
Emerging Technologies Architect IV
Chantilly, Virginia, United States
$133k-$208k/yr OnsiteFull Time
10+ YOEBachelor's degree and 10+ years of experience, including 4+ years evaluating emerging technologies, AI/ML architecture, or technology strategy. Requires US citizenship and active Top Secret clearance with SCI eligibility.
Artificial Intelligence (AI), machine learning, deep learning, large language models, computer vision, natural language processing, agentic AI, AWS, Microsoft Azure, GCP, AWS Solutions Architect, TOGAF
2mo
Save
Mark Applied
Hide
Content Automation and GenAI - Consulting and Delivery Manager
New York City or Milwaukee or Herndon or Dallas or Columbus or Kirkland or Irvine or Cincinnati or Cleveland or Oklahoma City or Austin or Albany or Chicago or Arlington or St. Petersburg or Hartford or Pittsburgh or St. Louis or Miami or Sacramento or Raleigh or Minneapolis or Mountain View or Scottsdale or Morristown or San Francisco or Denver or Boston or Philadelphia or Des Moines or Overland Park or Los Angeles or Charlotte or Walnut Creek or Carmel or Seattle or Houston or Atlanta or Redmond or Bentonville or Beaverton or Nashville or Detroit or San Diego
$80k-$266k/yr OnsiteFull Time
7+ YOE7+ years client-facing consulting or digital transformation experience; hands-on experience evaluating/applying content automation or GenAI tools and delivering GenAI POCs; strong understanding of LLM concepts, prompt engineering, safety/responsible AI, content operations, and team leadership.
OpenAI, Google Gemini, M365 Copilot, Writer, FireFly, ChatGPT, Adobe Workfront, AEM, CMS, DAM, PIM