27 ai evaluation engineer jobs at 21 companies in Peabody, MA
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
4+ YOEBachelor's plus 6 years or master's plus 4 years in AI/ML development; 6 years programming in Python, Go, Scala, or Java; experience building scalable AI systems and leading engineering teams.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, Google Cloud, Azure, C++, C#, Golang
2mo
Save
Mark Applied
Hide
Innovation Senior AI Engineer
Boston or Jersey City or New York City or Massachusetts or New York or New Jersey or United States or Poland
$210k-$260k/yr OnsiteFull Time
3+ YOE3+ years AI/ML engineering (5+ years product development), hands-on leader, expert Python, production LLM/RAG/agent experience, workflow orchestration, Azure/AWS, vector DBs, evaluation and responsible-AI practices.
Python, Agno, LangChain, LlamaIndex, Temporal, Prefect, Airflow, Azure, AWS, Pinecone, Weaviate, pgvector, LLM, RAG
1mo
Save
Mark Applied
Hide
Principal Engineer Tech Lead, Embodied AI & Off-Board Performance Evaluation
Boston or Las Vegas or Pittsburgh or United States
$200k-$275k/yr HybridFull Time
10+ YOE10+ years software/AI/AV experience, Bachelor's in CS/Engineering/Robotics, expertise with LLMs/VLMs, Python (expert), AV performance metrics, vector DBs, cloud infrastructure; strong analytical and system-design skills.
Large Language Models (LLMs), Vision-Language Models (VLMs), LanceDB, Ray, AWS, GCP, Python, C++
2mo
Save
Mark Applied
Hide
AI Engineer
Springfield or New York City or Boston
$141k-$185k/yr HybridFull Time
4+ YOE4+ years in data science/AI engineering with production ML/LLM experience, Python and SQL proficiency, ability to build and evaluate AI solutions, and a Bachelor’s in a quantitative field (Master’s preferred).
LLMs, Bedrock AgentCore, AWS Strands, Azure, MCP/A2A protocols, Python, SQL, vector databases, Degreed
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's degree plus 6 years or master's degree plus 4 years in AI/ML; 6 years programming with Python, Go, Scala, or Java; cloud AI deployment and team leadership preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, AWS, Google Cloud, Azure, C++, C#, Golang
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or Cambridge or San Jose or New York City
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's plus 6+ years (or Master's plus 4+ years) developing AI/ML systems; strong programming (Python, Go, Scala, Java); cloud AI deployment experience; leadership and research application skills.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
1w
Save
Mark Applied
Hide
Applied AI Engineer
Canada or San Francisco or Raleigh or Boston
HybridFull Time
Applied GenAI experience with prompt engineering, structured outputs, RAG, evaluation datasets, production LLM features, multimodal inputs, and model quality, cost, and latency optimization.
LLMX, Attachments, OCR, RAG
3mo
Save
Mark Applied
Hide
Chief Engineer - AI Test & Evaluation (T&E)
Cambridge or Arlington
$120k-$250k/yr OnsiteFull Time
10+ YOE5+ MgmtUS citizenship; security clearance; 10+ years in AI/ML R&D and T&E; master/PhD or equivalent; proven leadership; DoD/IC program experience.
AI, Machine Learning, Computer Vision, NLP, Large Language Models, AI Agents, Data Engineering, UI Systems
4w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1d
Save
Mark Applied
Hide
Oliver Wyman - Senior Lead Engineer
New York City or Boston or Washington or Chicago
$195k-$250k/yr HybridFull Time
Requires a quantitative or computational science background, large-scale production AI systems experience, modern AI programming fluency, agent platform knowledge, LLM design expertise, AI evaluation and governance skills, and strong mathematical grounding.
Azure Foundry, AWS Bedrock, Gemini Enterprise Agent Platform, Open-Source
1d
Save
Mark Applied
Hide
Oliver Wyman - Senior Lead Engineer
New York City or Boston or Chicago or Washington
$195k-$250k/yr HybridFull Time
Requires a quantitative or computational technical background and proven experience designing and deploying large-scale AI systems, agent platforms, production pipelines, evaluation frameworks, and LLM solutions.
Azure Foundry, AWS Bedrock, Gemini Enterprise Agent Platform
1w
Save
Mark Applied
Hide
Lead Decision Intelligence Engineer (AI) - NBA
United States or Boston
$129k-$178k/yr RemoteFull Time
6+ YOE1+ MgmtBachelor's degree and 6+ years of software, ML, AI, or decision intelligence experience, including technical leadership. Requires strong Python, agentic AI, enterprise AI platforms, decision modeling, evaluation, and team leadership.
Python, LangGraph, LangChain, AutoGen, CrewAI, Azure OpenAI, Azure AI Foundry, Vertex AI, Anthropic, OpenAI, Databricks, MLflow, Feature Store, Mosaic AI, Azure AI Search, LangSmith, OpenTelemetry, PromptFlow, Azure Monitor, Adobe Experience Platform (AEP), Salesforce, Drools, Decision Requirements Diagram (DRD), IBM ODM, DMN, OPA, Rego, Workday, HireVue
1mo
Save
Mark Applied
Hide
Agentic AI Architect - Intelligence Engineering (East)
Atlanta or Boston or Charlotte or Hartford or Miami or New Brunswick or New York City or Philadelphia or Raleigh or Washington
$108k-$218k/yr HybridFull Time
5+ YOE5+ years software engineering experience with 2+ years on generative/agentic AI; experience designing multi-agent systems, RAG, cloud AI platforms, Python and API frameworks, evaluation and governance for production AI systems.
Amazon Web Services, Azure, Google Cloud, AWS Bedrock, Azure AI Foundry, OpenAI Service, GCP Vertex AI, Gemini, Databricks, Lang Graph, Strands Agents SDK, OpenAI Agents SDK, Google ADK, Model Context Protocol (MCP), LangSmith, RAGAS, DeepEval, Claude Code, Cursor, Kiro, Python, FastAPI, Flask
1d
Save
Mark Applied
Hide
Manager/Senior Manager, Legal Technology and AI
Cambridge, Massachusetts, United States
$108k-$176k/yr HybridFull Time
Bachelor's degree in a technical field, AI/ML engineering experience, LLM/NLP knowledge, prompt engineering, RAG, fine-tuning, AI evaluation, legal operations exposure, legal technology familiarity, and strong communication skills.
AI/ML, large language models, natural language processing, prompt engineering, retrieval-augmented generation (RAG), CLM, document management, matter management, e-billing, GDPR, CCPA, CPRA
1mo
Save
Mark Applied
Hide
Senior / Staff Machine Learning Engineer, Applied AI
Cambridge or San Francisco
$180k-$336k/yr OnsiteFull Time
Strong ML engineering with LLM experience; expertise in training, adapting, evaluating, and deploying models; proficiency in Python and modern ML frameworks; ability to design experiments and debug model behavior.
Python, PyTorch, JAX, TensorFlow, Megatron-LM, TorchTitan, DeepSpeed, Ray
1d
Save
Mark Applied
Hide
Senior AI Data Engineering Specialist
Montréal or Andover
$115k-$162k/yr HybridFull Time
Requires data or knowledge engineering experience, data modeling and metadata management skills, Python or SQL pipeline proficiency, AI/LLM data preparation, and knowledge of embeddings, vector search, RAG, and evaluation datasets.
Python, SQL, Building Management System (BMS), RDF, OWL, Coursera
2w
Save
Mark Applied
Hide
Sr. Staff Engineer - Customer Agent
Boston, Massachusetts, United States
$216k-$324k/yr OnsiteFull Time
10+ YOE10+ years of software engineering experience with production systems at scale; expertise in AI-powered products, agentic systems, backend engineering, distributed systems, LLM applications, retrieval, orchestration, evaluation, and reliability.
LLM, AI
2w
Save
Mark Applied
Hide
Senior Data Scientist / ML Engineer (Gen AI)_Boston
Boston, Massachusetts, United States
$60k-$210k/yr HybridFull Time, Contract
Photon: An information technology organization developing enterprise artificial intelligence and machine learning solutions.
5+ YOERequires 5–10 years in data science or machine learning, strong Python, Pandas, NumPy, Scikit-learn, generative AI and LLM evaluation experience, RAG, embeddings, fine-tuning, and PyTorch or TensorFlow.
Python, Pandas, NumPy, Scikit-learn, PyTorch, TensorFlow, OpenAI, Claude, Gemini, Databricks, MLflow, Spark, Neo4j, Azure AI Foundry, AWS Bedrock, Kubernetes
3mo
Save
Mark Applied
Hide
Machine Learning Engineer
Boston, Massachusetts, United States
HybridFull Time
5+ YOEML/AI engineering with NLP focus; Python proficiency; data processing, model evaluation, and LLM experience; cloud; strong communication; security clearance a plus.
Python, NumPy, Pandas, Scikit-Learn, Transformers, Langraph, LlamaIndex, Ollama, Docker, Kubernetes
Explore Jobs
Expand Your Job Search
Top Cities Near Peabody for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Boston, MA
- AI Evaluation Engineer jobs in Worcester, MA
- AI Evaluation Engineer jobs in Lowell, MA
- AI Evaluation Engineer jobs in Cambridge, MA
- AI Evaluation Engineer jobs in Manchester, NH
- AI Evaluation Engineer jobs in Brockton, MA
- AI Evaluation Engineer jobs in Quincy, MA
- AI Evaluation Engineer jobs in Lynn, MA
- AI Evaluation Engineer jobs in Newton, MA
- AI Evaluation Engineer jobs in Nashua, NH
- AI Evaluation Engineer jobs in Somerville, MA
- AI Evaluation Engineer jobs in Lawrence, MA
- AI Evaluation Engineer jobs in Framingham, MA
- AI Evaluation Engineer jobs in Waltham, MA
- AI Evaluation Engineer jobs in Haverhill, MA
- AI Evaluation Engineer jobs in Malden, MA
- AI Evaluation Engineer jobs in Brookline, MA
- AI Evaluation Engineer jobs in Medford, MA
- AI Evaluation Engineer jobs in Taunton, MA
- AI Evaluation Engineer jobs in Revere, MA