30 software engineering evaluator jobs at 24 companies in Massachusetts
2mo
Save
Mark Applied
Hide
Director, Software Engineering – Research & Advanced Development
Bedford, Massachusetts, United States
$210k-$295k/yr OnsiteFull Time
10+ YOE3+ MgmtMaster's in robotics/CS/ML/AI, 10+ years software development experience (robotics/automation preferred), 3+ years technical leadership, strong Python/C++ skills, experience with real robotic systems and evaluating AI/ML robotics capabilities.
Python, C++, ROS, ROS2, PyTorch, Isaac Sim, Isaac Lab, MuJoCo
5d
Save
Mark Applied
Hide
Senior Software Engineer - Agents
United States or Boston
$180k-$220k/yr RemoteFull Time
4+ YOE4+ years of software engineering experience, including 1+ year building customer-facing agentic systems in production. Requires strong backend fundamentals, agentic architecture, production RAG, evaluations, observability, and AWS or GCP experience.
Python, Java, Go, Rust, TRACE Graph, RAG, AWS, GCP, LLM
1mo
Save
Mark Applied
Hide
Agentic AI Architect - Intelligence Engineering (East)
Atlanta or Boston or Charlotte or Hartford or Miami or New Brunswick or New York City or Philadelphia or Raleigh or Washington
$108k-$218k/yr HybridFull Time
5+ YOE5+ years software engineering experience with 2+ years on generative/agentic AI; experience designing multi-agent systems, RAG, cloud AI platforms, Python and API frameworks, evaluation and governance for production AI systems.
Amazon Web Services, Azure, Google Cloud, AWS Bedrock, Azure AI Foundry, OpenAI Service, GCP Vertex AI, Gemini, Databricks, Lang Graph, Strands Agents SDK, OpenAI Agents SDK, Google ADK, Model Context Protocol (MCP), LangSmith, RAGAS, DeepEval, Claude Code, Cursor, Kiro, Python, FastAPI, Flask
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
1mo
Save
Mark Applied
Hide
Senior AI Security Automation Engineer
Quincy or Toronto or Princeton or Boston or Clifton or Austin
$120k-$203k/yr HybridFull Time
7+ YOE7+ years software engineering, 3+ years building AI/ML/LLM solutions; expertise in cloud-native architectures, RAG, AI evaluation/observability, CI/CD, security, and mentoring engineers.
AWS, Azure, GCP, Jenkins, Harness, Spinnaker, Argo CD, Python, JavaScript/TypeScript, Rust, Go (Golang), Bash, PowerShell, LangGraph, Semantic Kernel, CrewAI, AutoGen, LangChain, Databricks, Snowflake, Spark, Kafka, Delta Lake, Iceberg, Airflow, vector databases
1w
Save
Mark Applied
Hide
Sr. Staff Engineer - Customer Agent
Boston, Massachusetts, United States
$216k-$324k/yr OnsiteFull Time
10+ YOE10+ years of software engineering experience with production systems at scale; expertise in AI-powered products, agentic systems, backend engineering, distributed systems, LLM applications, retrieval, orchestration, evaluation, and reliability.
LLM, AI
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1mo
Save
Mark Applied
Hide
Member of Technical Staff – Engineer, Policy Evaluation & Experiment Infrastructure
Cambridge or San Francisco
$152k-$203k/yr OnsiteFull Time
Experience building reliable evaluation tooling for robot policies, strong software fundamentals, hands-on robot or policy experience, interest in ML systems, and ability to deliver end-to-end.
Python
2mo
Save
Mark Applied
Hide
Test & Evaluation Engineer
Quincy, Massachusetts, United States
$146k-$194k/yr OnsiteFull Time
10+ YOE7+ MgmtBachelor's in engineering (or equivalent experience), 10+ years systems engineering/integration and testing, 7+ years lead test conductor experience, Top Secret clearance eligibility, maritime rules knowledge, Linux and software debugging experience, travel and extended hours ability.
Lattice OS, Linux, Go, Java, C++, Python, JavaScript, MIL-STD-1553, MIL-STD-1760, AS8775, AS5653
2h
Save
Mark Applied
Hide
INSTRUMENTATION CALIBRATION TECH
Caguas or Boston or United States or Puerto Rico or Dominican Republic or Mexico or Germany or Canada or South America
OnsiteFull Time
Associate degree in instrumentation technology or electronics; pharmaceutical calibration experience, tolerance evaluation, cGMP knowledge, and experience with calibration software and process instrumentation required.
Blue Mountain (RAM), Calman, Procal
1mo
Save
Mark Applied
Hide
Senior AI Platform Engineer
San Francisco or Alabama or Arkansas or California or Colorado or Florida or Georgia or Illinois or Kentucky or Massachusetts or Michigan or Montana or Missouri or Nevada or New Jersey or New York or North Carolina or Oregon or Pennsylvania or Texas or Washington or Utah or Virginia or Wisconsin
$156k-$231k/yr HybridFull Time
3+ YOE3+ years building production software or ML/data systems; strong Python and systems/API design; experience with LLM retrieval/RAG, agents, evaluation, Databricks/MLflow and modern data stacks.
Python, PySpark, dbt, Databricks, Delta Lake, Unity Catalog, MLflow, Astronomer, Airflow, Claude, GitHub, Shortcut, Notion, pgvector, Pinecone, Weaviate, LangGraph, MCP
2mo
Save
Mark Applied
Hide
Senior Engineer, Product Applications
Wilmington, Massachusetts, United States
$124k-$166k/yr HybridFull Time
2+ YOEMaster's in EE/Electronics/Computer Engineering required, 2+ years apps engineering experience with hardware/software/system testing and debugging in power, mixed-signal or electronic products; expertise with analog circuitry, DC-DC converters, ADCs/DACs, I2C/SPI, microcontrollers, PCB layout; programming in C, C#, Visual Basic, Python, MATLAB or L...
C, C#, Visual Basic, Python, MATLAB, LabView, SPICE, oscilloscopes, power supplies, DMMs, I2C, SPI, ADCs, DACs
1mo
Save
Mark Applied
Hide
Research Engineer
Cambridge or London
OnsiteFull Time
Use LLMs and agents to build software and systems, self-host and evaluate open-source models, strong Python, version control and testing, and ability to work across unfamiliar codebases.
LLMs, Python, AWS, GCP
1mo
Save
Mark Applied
Hide
Senior Forward Deployed AI Engineer (GenAI, AWS)
New Jersey or New York or Pennsylvania or North Carolina or Connecticut or Massachusetts or Virginia
RemoteFull Time
Provectus: Builds and deploys custom artificial intelligence and data solutions.
8+ YOE8+ years building production software, hands-on coding (Python/TypeScript), cloud-native AWS delivery, shipped GenAI/LLM systems to production, built evaluation suites, strong stakeholder communication and domain learning ability.
Claude, Claude Code, Cowork, AWS, Python, TypeScript, Kubernetes, ECS, PyTorch, SageMaker, MLflow, Neo4j, AWS Neptune, AWS CDK, CloudFormation, Terraform, ChatGPT
3mo
Save
Mark Applied
Hide
Machine Learning Engineer
Boston, Massachusetts, United States
HybridFull Time
5+ YOEML/AI engineering with NLP focus; Python proficiency; data processing, model evaluation, and LLM experience; cloud; strong communication; security clearance a plus.
Python, NumPy, Pandas, Scikit-Learn, Transformers, Langraph, LlamaIndex, Ollama, Docker, Kubernetes
2mo
Save
Mark Applied
Hide
AI Engineer
Springfield or New York City or Boston
$141k-$185k/yr HybridFull Time
4+ YOE4+ years in data science/AI engineering with production ML/LLM experience, Python and SQL proficiency, ability to build and evaluate AI solutions, and a Bachelor’s in a quantitative field (Master’s preferred).
LLMs, Bedrock AgentCore, AWS Strands, Azure, MCP/A2A protocols, Python, SQL, vector databases, Degreed
6d
Save
Mark Applied
Hide
Lead Decision Intelligence Engineer (AI) - NBA
United States or Boston
$129k-$178k/yr RemoteFull Time
6+ YOE1+ MgmtBachelor's degree and 6+ years of software, ML, AI, or decision intelligence experience, including technical leadership. Requires strong Python, agentic AI, enterprise AI platforms, decision modeling, evaluation, and team leadership.
Python, LangGraph, LangChain, AutoGen, CrewAI, Azure OpenAI, Azure AI Foundry, Vertex AI, Anthropic, OpenAI, Databricks, MLflow, Feature Store, Mosaic AI, Azure AI Search, LangSmith, OpenTelemetry, PromptFlow, Azure Monitor, Adobe Experience Platform (AEP), Salesforce, Drools, Decision Requirements Diagram (DRD), IBM ODM, DMN, OPA, Rego, Workday, HireVue
1mo
Save
Mark Applied
Hide
Senior / Staff Machine Learning Engineer, Applied AI
Cambridge or San Francisco
$180k-$336k/yr OnsiteFull Time
Strong ML engineering with LLM experience; expertise in training, adapting, evaluating, and deploying models; proficiency in Python and modern ML frameworks; ability to design experiments and debug model behavior.
Python, PyTorch, JAX, TensorFlow, Megatron-LM, TorchTitan, DeepSpeed, Ray
2w
Save
Mark Applied
Hide
Machine Learning Engineer
Massachusetts, United States
HybridFull Time
4+ YOERequires 4+ years in machine learning engineering, production deployments, PyTorch, Python, computer vision, GPU infrastructure, ML evaluation, and data pipeline tooling; Azure and re-identification experience required.
YOLOv8, RT-DETR, Triton, TorchServe, MLflow, Weights & Biases, PyTorch, Ultralytics, AWS SageMaker, GCP Vertex AI, Python, Kafka, Airflow, LSTMs, Prophet, XGBoost, Microsoft Azure
1mo
Save
Mark Applied
Hide
Principal Engineer Tech Lead, Embodied AI & Off-Board Performance Evaluation
Boston or Las Vegas or Pittsburgh or United States
$200k-$275k/yr HybridFull Time
10+ YOE10+ years software/AI/AV experience, Bachelor's in CS/Engineering/Robotics, expertise with LLMs/VLMs, Python (expert), AV performance metrics, vector DBs, cloud infrastructure; strong analytical and system-design skills.
Large Language Models (LLMs), Vision-Language Models (VLMs), LanceDB, Ray, AWS, GCP, Python, C++
Explore Jobs
Expand Your Job Search
Top Cities in Massachusetts for Software Engineering Evaluator
- Software Engineering Evaluator jobs in Boston, MA
- Software Engineering Evaluator jobs in Worcester, MA
- Software Engineering Evaluator jobs in Springfield, MA
- Software Engineering Evaluator jobs in Lowell, MA
- Software Engineering Evaluator jobs in Cambridge, MA
- Software Engineering Evaluator jobs in New Bedford, MA
- Software Engineering Evaluator jobs in Brockton, MA
- Software Engineering Evaluator jobs in Fall River, MA
- Software Engineering Evaluator jobs in Quincy, MA
- Software Engineering Evaluator jobs in Lynn, MA
- Software Engineering Evaluator jobs in Newton, MA
- Software Engineering Evaluator jobs in Somerville, MA
- Software Engineering Evaluator jobs in Lawrence, MA
- Software Engineering Evaluator jobs in Framingham, MA
- Software Engineering Evaluator jobs in Waltham, MA
- Software Engineering Evaluator jobs in Haverhill, MA
- Software Engineering Evaluator jobs in Malden, MA
- Software Engineering Evaluator jobs in Brookline, MA
- Software Engineering Evaluator jobs in Medford, MA
- Software Engineering Evaluator jobs in Taunton, MA