24 software engineering evaluator jobs at 21 companies in Framingham, MA
2mo
Save
Mark Applied
Hide
Director, Software Engineering – Research & Advanced Development
Bedford, Massachusetts, United States
$210k-$295k/yr OnsiteFull Time
10+ YOE3+ MgmtMaster's in robotics/CS/ML/AI, 10+ years software development experience (robotics/automation preferred), 3+ years technical leadership, strong Python/C++ skills, experience with real robotic systems and evaluating AI/ML robotics capabilities.
Python, C++, ROS, ROS2, PyTorch, Isaac Sim, Isaac Lab, MuJoCo
5d
Save
Mark Applied
Hide
Senior Software Engineer - Agents
United States or Boston
$180k-$220k/yr RemoteFull Time
4+ YOE4+ years of software engineering experience, including 1+ year building customer-facing agentic systems in production. Requires strong backend fundamentals, agentic architecture, production RAG, evaluations, observability, and AWS or GCP experience.
Python, Java, Go, Rust, TRACE Graph, RAG, AWS, GCP, LLM
1mo
Save
Mark Applied
Hide
Agentic AI Architect - Intelligence Engineering (East)
Atlanta or Boston or Charlotte or Hartford or Miami or New Brunswick or New York City or Philadelphia or Raleigh or Washington
$108k-$218k/yr HybridFull Time
5+ YOE5+ years software engineering experience with 2+ years on generative/agentic AI; experience designing multi-agent systems, RAG, cloud AI platforms, Python and API frameworks, evaluation and governance for production AI systems.
Amazon Web Services, Azure, Google Cloud, AWS Bedrock, Azure AI Foundry, OpenAI Service, GCP Vertex AI, Gemini, Databricks, Lang Graph, Strands Agents SDK, OpenAI Agents SDK, Google ADK, Model Context Protocol (MCP), LangSmith, RAGAS, DeepEval, Claude Code, Cursor, Kiro, Python, FastAPI, Flask
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
1mo
Save
Mark Applied
Hide
Senior AI Security Automation Engineer
Quincy or Toronto or Princeton or Boston or Clifton or Austin
$120k-$203k/yr HybridFull Time
7+ YOE7+ years software engineering, 3+ years building AI/ML/LLM solutions; expertise in cloud-native architectures, RAG, AI evaluation/observability, CI/CD, security, and mentoring engineers.
AWS, Azure, GCP, Jenkins, Harness, Spinnaker, Argo CD, Python, JavaScript/TypeScript, Rust, Go (Golang), Bash, PowerShell, LangGraph, Semantic Kernel, CrewAI, AutoGen, LangChain, Databricks, Snowflake, Spark, Kafka, Delta Lake, Iceberg, Airflow, vector databases
1w
Save
Mark Applied
Hide
Sr. Staff Engineer - Customer Agent
Boston, Massachusetts, United States
$216k-$324k/yr OnsiteFull Time
10+ YOE10+ years of software engineering experience with production systems at scale; expertise in AI-powered products, agentic systems, backend engineering, distributed systems, LLM applications, retrieval, orchestration, evaluation, and reliability.
LLM, AI
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1mo
Save
Mark Applied
Hide
Member of Technical Staff – Engineer, Policy Evaluation & Experiment Infrastructure
Cambridge or San Francisco
$152k-$203k/yr OnsiteFull Time
Experience building reliable evaluation tooling for robot policies, strong software fundamentals, hands-on robot or policy experience, interest in ML systems, and ability to deliver end-to-end.
Python
2mo
Save
Mark Applied
Hide
Test & Evaluation Engineer
Quincy, Massachusetts, United States
$146k-$194k/yr OnsiteFull Time
10+ YOE7+ MgmtBachelor's in engineering (or equivalent experience), 10+ years systems engineering/integration and testing, 7+ years lead test conductor experience, Top Secret clearance eligibility, maritime rules knowledge, Linux and software debugging experience, travel and extended hours ability.
Lattice OS, Linux, Go, Java, C++, Python, JavaScript, MIL-STD-1553, MIL-STD-1760, AS8775, AS5653
2mo
Save
Mark Applied
Hide
Senior Engineer, Product Applications
Wilmington, Massachusetts, United States
$124k-$166k/yr HybridFull Time
2+ YOEMaster's in EE/Electronics/Computer Engineering required, 2+ years apps engineering experience with hardware/software/system testing and debugging in power, mixed-signal or electronic products; expertise with analog circuitry, DC-DC converters, ADCs/DACs, I2C/SPI, microcontrollers, PCB layout; programming in C, C#, Visual Basic, Python, MATLAB or L...
C, C#, Visual Basic, Python, MATLAB, LabView, SPICE, oscilloscopes, power supplies, DMMs, I2C, SPI, ADCs, DACs
1mo
Save
Mark Applied
Hide
Research Engineer
Cambridge or London
OnsiteFull Time
Use LLMs and agents to build software and systems, self-host and evaluate open-source models, strong Python, version control and testing, and ability to work across unfamiliar codebases.
LLMs, Python, AWS, GCP
3mo
Save
Mark Applied
Hide
Machine Learning Engineer
Boston, Massachusetts, United States
HybridFull Time
5+ YOEML/AI engineering with NLP focus; Python proficiency; data processing, model evaluation, and LLM experience; cloud; strong communication; security clearance a plus.
Python, NumPy, Pandas, Scikit-Learn, Transformers, Langraph, LlamaIndex, Ollama, Docker, Kubernetes
2mo
Save
Mark Applied
Hide
AI Engineer
Springfield or New York City or Boston
$141k-$185k/yr HybridFull Time
4+ YOE4+ years in data science/AI engineering with production ML/LLM experience, Python and SQL proficiency, ability to build and evaluate AI solutions, and a Bachelor’s in a quantitative field (Master’s preferred).
LLMs, Bedrock AgentCore, AWS Strands, Azure, MCP/A2A protocols, Python, SQL, vector databases, Degreed
6d
Save
Mark Applied
Hide
Lead Decision Intelligence Engineer (AI) - NBA
United States or Boston
$129k-$178k/yr RemoteFull Time
6+ YOE1+ MgmtBachelor's degree and 6+ years of software, ML, AI, or decision intelligence experience, including technical leadership. Requires strong Python, agentic AI, enterprise AI platforms, decision modeling, evaluation, and team leadership.
Python, LangGraph, LangChain, AutoGen, CrewAI, Azure OpenAI, Azure AI Foundry, Vertex AI, Anthropic, OpenAI, Databricks, MLflow, Feature Store, Mosaic AI, Azure AI Search, LangSmith, OpenTelemetry, PromptFlow, Azure Monitor, Adobe Experience Platform (AEP), Salesforce, Drools, Decision Requirements Diagram (DRD), IBM ODM, DMN, OPA, Rego, Workday, HireVue
1mo
Save
Mark Applied
Hide
Senior / Staff Machine Learning Engineer, Applied AI
Cambridge or San Francisco
$180k-$336k/yr OnsiteFull Time
Strong ML engineering with LLM experience; expertise in training, adapting, evaluating, and deploying models; proficiency in Python and modern ML frameworks; ability to design experiments and debug model behavior.
Python, PyTorch, JAX, TensorFlow, Megatron-LM, TorchTitan, DeepSpeed, Ray
1mo
Save
Mark Applied
Hide
Principal Engineer Tech Lead, Embodied AI & Off-Board Performance Evaluation
Boston or Las Vegas or Pittsburgh or United States
$200k-$275k/yr HybridFull Time
10+ YOE10+ years software/AI/AV experience, Bachelor's in CS/Engineering/Robotics, expertise with LLMs/VLMs, Python (expert), AV performance metrics, vector DBs, cloud infrastructure; strong analytical and system-design skills.
Large Language Models (LLMs), Vision-Language Models (VLMs), LanceDB, Ray, AWS, GCP, Python, C++
3mo
Save
Mark Applied
Hide
Sales Engineer
Boston, Massachusetts, United States
OnsiteFull Time
Deliver platform demos, articulate value, and engage customers to drive technical evaluation.
Terraform, Cloud platforms
1w
Save
Mark Applied
Hide
Oliver Wyman - Senior Lead AI Engineer
New York City or Boston or Toronto or Montreal
$195k-$250k/yr HybridFull Time
Requires quantitative or computational science background, large-scale production AI experience, modern AI programming fluency, agentic platform knowledge, LLM systems expertise, evaluation and observability experience, and mathematical grounding in AI/ML.
Azure Foundry, AWS Bedrock, Gemini Enterprise Agent Platform
2mo
Save
Mark Applied
Hide
Staff Agentic ML Engineer - Photoshop
San Jose or Waltham or San Francisco or Seattle or Los Angeles or New York City or California
$190k-$346k/yr OnsiteFull Time
8+ YOEMaster's or PhD in CS/ML/Data Science, 8+ years ML experience, expertise in LLMs, fine-tuning (SFT, RLHF, DPO, PEFT/LoRA), agent system development, evaluation frameworks; proficiency with PyTorch and agentic frameworks; strong communication and problem-solving.
PyTorch, LangChain, LangGraph, MCP, A2A, Agent Development Kit (ADK), Claude Code, Codex, Cursor
4w
Save
Mark Applied
Hide
Security Engineer, Correlation and Response, AWS Security Hub
Boston or Seattle
$159k-$202k/yr OnsiteFull Time
3+ YOE3+ years programming and security experience; threat evaluation, scripting, cloud software development, knowledge of web protocols, and security analysis for AWS/Azure/GCP.
Python, Perl, Bash, PowerShell, Java, AWS Developer Tools, Linux, Unix, AWS, Azure, GCP, Ruby, Go, Swift, .NET, C++
Explore Jobs
Expand Your Job Search
Top Cities Near Framingham for Software Engineering Evaluator
- Software Engineering Evaluator jobs in Boston, MA
- Software Engineering Evaluator jobs in Worcester, MA
- Software Engineering Evaluator jobs in Providence, RI
- Software Engineering Evaluator jobs in Lowell, MA
- Software Engineering Evaluator jobs in Cambridge, MA
- Software Engineering Evaluator jobs in Manchester, NH
- Software Engineering Evaluator jobs in Brockton, MA
- Software Engineering Evaluator jobs in Fall River, MA
- Software Engineering Evaluator jobs in Quincy, MA
- Software Engineering Evaluator jobs in Lynn, MA
- Software Engineering Evaluator jobs in Newton, MA
- Software Engineering Evaluator jobs in Nashua, NH
- Software Engineering Evaluator jobs in Warwick, RI
- Software Engineering Evaluator jobs in Cranston, RI
- Software Engineering Evaluator jobs in Somerville, MA
- Software Engineering Evaluator jobs in Lawrence, MA
- Software Engineering Evaluator jobs in Pawtucket, RI
- Software Engineering Evaluator jobs in Waltham, MA
- Software Engineering Evaluator jobs in Haverhill, MA
- Software Engineering Evaluator jobs in Malden, MA