HiringCafe
Join our community

523 evaluations engineer jobs at 254 companies in San Mateo, CA

1mo
Save
Mark Applied
Hide
Evaluations Engineer
San Francisco, California, United States
$130k-$165k/yr OnsiteFull Time
Vals AI
Vals AI: Building enterprise benchmarks for evaluating LLM performance.
Strong engineering fundamentals, professional Python expertise, familiarity with LLMs, Git workflow experience, ability to analyze model error modes and work with cross-functional teams. In-person in San Francisco; relocation/transportation support provided.
Docent, Python, Django, React, AWS, CDK, Git
Job PostingView all
2mo
Save
Mark Applied
Hide
Field Evaluations Engineer - Northern California
Fremont or San Jose or Brea or California
$97k-$120k/yr FieldFull Time
UL Solutions
UL SolutionsNYSE: ULS: Provides safety science testing, inspection, and certification services globally.
8+ YOEBachelor's in an ABET-accredited engineering field, 8 years related experience, expert knowledge of NEC/building codes, electrical test equipment, hazard-based engineering, frequent travel and ability to work remotely.
Electrical Test Equipment
Job PostingView all
3mo
Save
Mark Applied
Hide
AI Evaluations Engineer
San Francisco, California, United States
$160k-$210k/yr RemoteFull Time
Ellipsis Health
Ellipsis Health: AI-powered voice agent for healthcare care management.
5+ YOE5+ years software engineering; backend systems; production infrastructure; CI/CD; observability; debugging; collaboration across teams.
Python, TypeScript, Java, CI/CD, APIs, Data Pipelines, Observability Tools
Job PostingView all
1mo
Save
Mark Applied
Hide
Evaluations Engineering - Member of Technical Staff
San Francisco, California, United States
$200k-$400k/yr OnsiteFull Time
Simile
Simile: Building AI simulations to predict human behavior at scale.
Several years building production backend and data systems, experience with evaluation pipelines, ML/LLM fluency, strong system design and communication skills.
Job PostingView all
1mo
Save
Mark Applied
Hide
Research Engineer - Evaluations
Redwood City, California, United States
HybridFull Time
Luma AI
Luma AI: Develops multimodal AI for video generation and creative production.
5+ YOE5+ years building ML evaluation systems or large-scale model pipelines; master's/PhD or equivalent; experience with visual data; proficiency in Python and PyTorch/JAX/TensorFlow; strong software engineering and distributed systems skills.
Python, PyTorch, JAX, TensorFlow, CI/CD
Job PostingView all
4w
Save
Mark Applied
Hide
Member of Technical Staff - Research Software Engineer - Safety Evaluations Infrastructure
New York City or San Francisco or London
OnsiteFull Time
Reflection AI
Reflection AI: Developing open-source frontier artificial intelligence models and coding agents.
Experience building secure, isolated infrastructure for sensitive model evaluations; strong Python skills; security engineering fundamentals; data pipeline and sandboxing experience.
Python
Job PostingView all
3mo
Save
Mark Applied
Hide
Machine Learning Engineer, Model Evaluations (Speech LLM) - San Francisco
San Francisco, California, United States
$180k-$270k/yr HybridFull Time
Plaud
Plaud: Develops AI-powered voice recorders and automated transcription software.
Python software engineering; building distributed systems, data pipelines, and evaluation harnesses at scale; partner with ML researchers to define benchmarks; build dashboards and monitor model health; debug mid-training anomalies; communicate results clearly.
Python, Distributed systems, Data pipelines, Evaluation harnesses, Dashboards, Weighs & Biases, MLflow
Job PostingView all
2mo
Save
Mark Applied
Hide
Staff Software Engineer, Quantitative Evaluations
Mountain View or San Francisco
$251k-$310k/yr HybridFull Time
Waymo
Waymo: Autonomous driving technology for ride-hailing and logistics.
4+ YOEBS in a quantitative field; 4-7 years experience; strong C++, Python, SQL skills; data analysis; building data pipelines; ML familiarity; strong coding standards.
C++, Python, SQL, Machine Learning, Data Processing, Statistics
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
Santa Clara, California, United States
$201k-$352k/yr OnsiteFull Time
ServiceNow
ServiceNowNYSE: NOW: Provides a cloud platform for automating enterprise digital workflows.
8+ YOE3+ Mgmt8+ years software/ML engineering experience with 3+ years managing AI/ML teams; expertise in LLM evaluation, benchmarking, Python, SQL, Pandas, NumPy, and ML evaluation metrics.
Python, SQL, Pandas, NumPy, ROUGE, BLEU, G-Eval, OpenAI, Anthropic, Google
Job PostingView all
2mo
Save
Mark Applied
Hide
AI Engineer, Evaluation
San Francisco or New York City
$150k-$250k/yr HybridFull Time
Distyl AI
Distyl AI: Builds AI-native orchestration platforms for enterprise operations.
2+ YOE2+ years software engineering, strong Python, experience with evaluation- or experiment-driven development, ability to encode human judgment into tests/graders, systems-oriented mindset, and willingness to travel 10–50%.
Python, LLM
Job PostingView all
1w
Save
Mark Applied
Hide
Electrical Engineer
Sunnyvale, California, United States
$74k-$116k/yr OnsiteFull Time
Abbott
AbbottNYSE: ABT: Manufactures medical devices, diagnostics, and nutritional health products.
3+ YOEBachelor's degree in electrical, electronic, or related engineering; 3+ years designing and evaluating PCBs, fixtures, and tools; experience with testing, documentation, FDA requirements, and engineering laboratory equipment.
Printed Circuit Boards (PCBs), oscilloscopes, spectrum analyzers, arbitrary waveform generators, function generators, LCR meters, power supplies, embedded systems, DSP, SPICE, Matlab, COMSOL, spreadsheet, word processing, database management
Job PostingView all
2w
Save
Mark Applied
Hide
Founding Research Engineer
New York City or San Francisco
$215k-$330k/yr OnsiteFull Time
Ambral
Ambral: AI-powered account managers for enterprise customer success teams.
4+ YOE4+ years building production software or ML systems, including 2+ years with RL environments or evaluation/agent infrastructure; strong software engineering, data processing, reproducibility, and evaluation skills.
Job PostingView all
3mo
Save
Mark Applied
Hide
Senior Engineer, Design Evaluation Engineering
San Jose, California, United States
$157k-$188k/yr OnsiteFull Time
Analog Devices
Analog DevicesNASDAQ: ADI: Designs and manufactures semiconductors for signal processing and power management.
1+ YOEMaster’s degree in Electrical Engineering and 1 year of relevant design evaluation experience; proficient in C#/C++, Python, and PCB design tools; able to automate tests and analyze results.
C#, C++, Python, Visual Studio, OrCAD, Cadence Allegro, Microsoft Office, PCB schematic capture, PCB layout
Job PostingView all
1w
Save
Mark Applied
Hide
AI Engineer – Algorithm Evaluation & Agentic Systems
Sunnyvale, California, United States
OnsiteFull Time
Apple
AppleNASDAQ: AAPL: Designs and sells consumer electronics, software, and online services.
Experience evaluating AI algorithms and designing agentic systems, with system-level reasoning, metric definition, engineering rigor, and clear technical communication.
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
Santa Clara, California, United States
$201k-$352k/yr OnsiteFull Time
ServiceNow
ServiceNowNYSE: NOW: Enterprise cloud platform for digital workflow automation.
8+ YOE3+ MgmtRequires 8+ years of software engineering or machine learning experience, 3+ years leading AI/ML teams, Python, SQL, Pandas, NumPy, AI evaluation metrics, MLOps platforms, and a bachelor's degree in a quantitative field.
OpenAI, Anthropic, Google, ROUGE, BLEU, G-Eval, Python, SQL, Pandas, NumPy, MLOps, Knowledge Graphs, CMDBs, RAG
Job PostingView all
4w
Save
Mark Applied
Hide
Evaluation and ML Systems Engineer, AI Safety and Security Engineering
Santa Clara or North Carolina or Tennessee or New York or California or Florida
$152k-$242k/yr HybridFull Time
NVIDIA
NVIDIANASDAQ: NVDA: Designs graphics processing units and artificial intelligence hardware.
5+ YOE5+ years ML engineering or evaluation experience, bachelor's degree or equivalent, benchmark and metrics design, Python engineering, experiment tracking, and statistical evaluation skills.
Python
Job PostingView all
1w
Save
Mark Applied
Hide
Electrical Engineer
Sunnyvale, California, United States
$74k-$116k/yr OnsiteFull Time
Abbott
AbbottNYSE: ABT: Provides medical devices, diagnostics, and science-based nutritional products.
3+ YOEBachelor's degree in electrical, electronic, or related engineering and 3+ years of PCB, fixture, and tool construction and evaluation experience. Requires FDA compliance, testing, documentation, communication, and analytical skills.
Printed Circuit Boards (PCBs), oscilloscopes, spectrum analyzers, arbitrary waveform generators, function generators, LCR meters, power supplies, embedded systems, DSP, SPICE, Matlab, COMSOL
Job PostingView all
1w
Save
Mark Applied
Hide
AI Engineer
Mountain View, California, United States
OnsiteFull Time
AI Fund
AI Fund: Builds and invests in artificial intelligence startups.
3+ YOERequires 3+ years of software engineering experience, production LLM systems experience, strong Python and/or TypeScript/Node skills, and experience with LLM APIs, agentic workflows, evaluation, and end-to-end services.
Claude, OpenAI, Python, TypeScript, Node
Job PostingView all
3mo
Save
Mark Applied
Hide
Lead AI Engineer — Productivity Systems
Durham or Miami or Palo Alto or Washington
$12k-$18k/mo HybridFull Time
Nubank
NubankNYSE: NU: Digital financial platform offering banking, credit, and investment services.
Proven experience shipping LLM-powered systems in production; strong prompt/context engineering, evaluation and guardrail development, and software engineering skills (Python/TypeScript/Clojure).
Python, TypeScript, Clojure, n8n, Zapier, AWS, Claude Code, Cursor, Copilot, Slack, Google Workspace, Jira, Terraform, RAG, MCP
Job PostingView all
2w
Save
Mark Applied
Hide
Forward Deployed Engineer
Mountain View or San Francisco
$160k-$220k/yr OnsiteFull Time
Granica
Granica: Provides an AI efficiency platform for optimizing enterprise training data.
5+ YOERequires 5+ years in sales engineering, solutions architecture, forward deployed or data engineering; expertise in data platforms, distributed systems, cloud infrastructure, SQL, and programming; enterprise evaluations and deployments experience.
Apache Spark, Apache Iceberg, Delta Lake, Trino, Presto, SQL, Large Tabular Models (LTMs)
Job PostingView all

Explore Jobs

Expand Your Job Search