27 ai evaluation engineer jobs at 19 companies in Warwick, RI
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
4+ YOEBachelor's plus 6 years or master's plus 4 years in AI/ML development; 6 years programming in Python, Go, Scala, or Java; experience building scalable AI systems and leading engineering teams.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, Google Cloud, Azure, C++, C#, Golang
2mo
Save
Mark Applied
Hide
Innovation Senior AI Engineer
Boston or Jersey City or New York City or Massachusetts or New York or New Jersey or United States or Poland
$210k-$260k/yr OnsiteFull Time
3+ YOE3+ years AI/ML engineering (5+ years product development), hands-on leader, expert Python, production LLM/RAG/agent experience, workflow orchestration, Azure/AWS, vector DBs, evaluation and responsible-AI practices.
Python, Agno, LangChain, LlamaIndex, Temporal, Prefect, Airflow, Azure, AWS, Pinecone, Weaviate, pgvector, LLM, RAG
1w
Save
Mark Applied
Hide
Oliver Wyman - Senior Lead AI Engineer
New York City or Boston or Toronto or Montreal
$195k-$250k/yr HybridFull Time
Expertise designing and deploying large-scale AI systems, agent platforms, LLMs, data engineering, evaluation, observability, mathematical modeling, and modern AI programming languages.
Azure Foundry, AWS Bedrock, Gemini Enterprise Agent Platform, LLM, Open-Source
1mo
Save
Mark Applied
Hide
Principal Engineer Tech Lead, Embodied AI & Off-Board Performance Evaluation
Boston or Las Vegas or Pittsburgh or United States
$200k-$275k/yr HybridFull Time
10+ YOE10+ years software/AI/AV experience, Bachelor's in CS/Engineering/Robotics, expertise with LLMs/VLMs, Python (expert), AV performance metrics, vector DBs, cloud infrastructure; strong analytical and system-design skills.
Large Language Models (LLMs), Vision-Language Models (VLMs), LanceDB, Ray, AWS, GCP, Python, C++
2mo
Save
Mark Applied
Hide
AI Engineer
Springfield or New York City or Boston
$141k-$185k/yr HybridFull Time
4+ YOE4+ years in data science/AI engineering with production ML/LLM experience, Python and SQL proficiency, ability to build and evaluate AI solutions, and a Bachelor’s in a quantitative field (Master’s preferred).
LLMs, Bedrock AgentCore, AWS Strands, Azure, MCP/A2A protocols, Python, SQL, vector databases, Degreed
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's degree plus 6 years or master's degree plus 4 years in AI/ML; 6 years programming with Python, Go, Scala, or Java; cloud AI deployment and team leadership preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, AWS, Google Cloud, Azure, C++, C#, Golang
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or Cambridge or San Jose or New York City
$230k-$286k/yr OnsiteFull Time
6+ YOEBachelor's plus 6+ years (or Master's plus 4+ years) developing AI/ML systems; strong programming (Python, Go, Scala, Java); cloud AI deployment experience; leadership and research application skills.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
1w
Save
Mark Applied
Hide
Applied AI Engineer
Canada or San Francisco or Raleigh or Boston
HybridFull Time
Applied GenAI experience with prompt engineering, structured outputs, RAG, evaluation datasets, production LLM features, multimodal inputs, and model quality, cost, and latency optimization.
LLMX, Attachments, OCR, RAG
3mo
Save
Mark Applied
Hide
Chief Engineer - AI Test & Evaluation (T&E)
Cambridge or Arlington
$120k-$250k/yr OnsiteFull Time
10+ YOE5+ MgmtUS citizenship; security clearance; 10+ years in AI/ML R&D and T&E; master/PhD or equivalent; proven leadership; DoD/IC program experience.
AI, Machine Learning, Computer Vision, NLP, Large Language Models, AI Agents, Data Engineering, UI Systems
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1w
Save
Mark Applied
Hide
Oliver Wyman - Senior Lead AI Engineer
New York City or Boston or Toronto or Montreal
$195k-$250k/yr HybridFull Time
Requires quantitative or computational science background, large-scale production AI experience, modern AI programming fluency, agentic platform knowledge, LLM systems expertise, evaluation and observability experience, and mathematical grounding in AI/ML.
Azure Foundry, AWS Bedrock, Gemini Enterprise Agent Platform
1w
Save
Mark Applied
Hide
Lead Decision Intelligence Engineer (AI) - NBA
United States or Boston
$129k-$178k/yr RemoteFull Time
6+ YOE1+ MgmtBachelor's degree and 6+ years of software, ML, AI, or decision intelligence experience, including technical leadership. Requires strong Python, agentic AI, enterprise AI platforms, decision modeling, evaluation, and team leadership.
Python, LangGraph, LangChain, AutoGen, CrewAI, Azure OpenAI, Azure AI Foundry, Vertex AI, Anthropic, OpenAI, Databricks, MLflow, Feature Store, Mosaic AI, Azure AI Search, LangSmith, OpenTelemetry, PromptFlow, Azure Monitor, Adobe Experience Platform (AEP), Salesforce, Drools, Decision Requirements Diagram (DRD), IBM ODM, DMN, OPA, Rego, Workday, HireVue
1mo
Save
Mark Applied
Hide
Agentic AI Architect - Intelligence Engineering (East)
Atlanta or Boston or Charlotte or Hartford or Miami or New Brunswick or New York City or Philadelphia or Raleigh or Washington
$108k-$218k/yr HybridFull Time
5+ YOE5+ years software engineering experience with 2+ years on generative/agentic AI; experience designing multi-agent systems, RAG, cloud AI platforms, Python and API frameworks, evaluation and governance for production AI systems.
Amazon Web Services, Azure, Google Cloud, AWS Bedrock, Azure AI Foundry, OpenAI Service, GCP Vertex AI, Gemini, Databricks, Lang Graph, Strands Agents SDK, OpenAI Agents SDK, Google ADK, Model Context Protocol (MCP), LangSmith, RAGAS, DeepEval, Claude Code, Cursor, Kiro, Python, FastAPI, Flask
20h
Save
Mark Applied
Hide
Manager/Senior Manager, Legal Technology and AI
Cambridge, Massachusetts, United States
$108k-$176k/yr HybridFull Time
Bachelor's degree in a technical field, AI/ML engineering experience, LLM/NLP knowledge, prompt engineering, RAG, fine-tuning, AI evaluation, legal operations exposure, legal technology familiarity, and strong communication skills.
AI/ML, large language models, natural language processing, prompt engineering, retrieval-augmented generation (RAG), CLM, document management, matter management, e-billing, GDPR, CCPA, CPRA
1mo
Save
Mark Applied
Hide
Senior / Staff Machine Learning Engineer, Applied AI
Cambridge or San Francisco
$180k-$336k/yr OnsiteFull Time
Strong ML engineering with LLM experience; expertise in training, adapting, evaluating, and deploying models; proficiency in Python and modern ML frameworks; ability to design experiments and debug model behavior.
Python, PyTorch, JAX, TensorFlow, Megatron-LM, TorchTitan, DeepSpeed, Ray
2w
Save
Mark Applied
Hide
Sr. Staff Engineer - Customer Agent
Boston, Massachusetts, United States
$216k-$324k/yr OnsiteFull Time
10+ YOE10+ years of software engineering experience with production systems at scale; expertise in AI-powered products, agentic systems, backend engineering, distributed systems, LLM applications, retrieval, orchestration, evaluation, and reliability.
LLM, AI
2w
Save
Mark Applied
Hide
Senior Data Scientist / ML Engineer (Gen AI)_Boston
Boston, Massachusetts, United States
$60k-$210k/yr HybridFull Time, Contract
Photon: An information technology organization developing enterprise artificial intelligence and machine learning solutions.
5+ YOERequires 5–10 years in data science or machine learning, strong Python, Pandas, NumPy, Scikit-learn, generative AI and LLM evaluation experience, RAG, embeddings, fine-tuning, and PyTorch or TensorFlow.
Python, Pandas, NumPy, Scikit-learn, PyTorch, TensorFlow, OpenAI, Claude, Gemini, Databricks, MLflow, Spark, Neo4j, Azure AI Foundry, AWS Bedrock, Kubernetes
3mo
Save
Mark Applied
Hide
Machine Learning Engineer
Boston, Massachusetts, United States
HybridFull Time
5+ YOEML/AI engineering with NLP focus; Python proficiency; data processing, model evaluation, and LLM experience; cloud; strong communication; security clearance a plus.
Python, NumPy, Pandas, Scikit-Learn, Transformers, Langraph, LlamaIndex, Ollama, Docker, Kubernetes
2w
Save
Mark Applied
Hide
Senior Product Manager, AI & Data Science Products
Seattle or Richmond or Austin or North Charleston or Philadelphia or San Francisco or Denver or Miami or Atlanta or Chicago or Augusta or Boston or Las Vegas or Trenton or New York or Raleigh or Portland
RemoteFull Time
3+ YOERequires 3+ years of product management or comparable experience, customer-facing AI or data product ownership, applied AI/ML knowledge, data science and engineering collaboration, experimentation, analytics, and quality evaluation skills.
AI, machine learning, APIs, MCP
Explore Jobs
Expand Your Job Search
Top Cities Near Warwick for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Boston, MA
- AI Evaluation Engineer jobs in Worcester, MA
- AI Evaluation Engineer jobs in Providence, RI
- AI Evaluation Engineer jobs in Cambridge, MA
- AI Evaluation Engineer jobs in New Bedford, MA
- AI Evaluation Engineer jobs in Brockton, MA
- AI Evaluation Engineer jobs in Fall River, MA
- AI Evaluation Engineer jobs in Quincy, MA
- AI Evaluation Engineer jobs in Newton, MA
- AI Evaluation Engineer jobs in Cranston, RI
- AI Evaluation Engineer jobs in Pawtucket, RI
- AI Evaluation Engineer jobs in Framingham, MA
- AI Evaluation Engineer jobs in Waltham, MA
- AI Evaluation Engineer jobs in Brookline, MA
- AI Evaluation Engineer jobs in Taunton, MA
- AI Evaluation Engineer jobs in East Providence, RI
- AI Evaluation Engineer jobs in Attleboro, MA
- AI Evaluation Engineer jobs in Woonsocket, RI
- AI Evaluation Engineer jobs in Norwich, CT
- AI Evaluation Engineer jobs in Marlborough, MA