HiringCafe
Join our community

711 evaluations engineer jobs at 331 companies in California

1mo
Save
Mark Applied
Hide
Evaluations Engineer
San Francisco, California, United States
$130k-$165k/yr OnsiteFull Time
Vals AI
Vals AI: Building enterprise benchmarks for evaluating LLM performance.
Strong engineering fundamentals, professional Python expertise, familiarity with LLMs, Git workflow experience, ability to analyze model error modes and work with cross-functional teams. In-person in San Francisco; relocation/transportation support provided.
Docent, Python, Django, React, AWS, CDK, Git
Job PostingView all
2mo
Save
Mark Applied
Hide
Field Evaluations Engineer - West US Region
California or San Diego or Brea or Vancouver or Seattle
$97k-$120k/yr FieldFull Time
UL Solutions
UL SolutionsNYSE: ULS: Provides safety science testing, inspection, and certification services globally.
8+ YOEABET-accredited bachelor's in engineering or related field, 8+ years related experience, expert knowledge of NEC/building codes and field evaluations, experience with electrical test equipment, frequent travel and airport proximity required.
Electrical Test Equipment, NEC
Job PostingView all
3mo
Save
Mark Applied
Hide
AI Evaluations Engineer
San Francisco, California, United States
$160k-$210k/yr RemoteFull Time
Ellipsis Health
Ellipsis Health: AI-powered voice agent for healthcare care management.
5+ YOE5+ years software engineering; backend systems; production infrastructure; CI/CD; observability; debugging; collaboration across teams.
Python, TypeScript, Java, CI/CD, APIs, Data Pipelines, Observability Tools
Job PostingView all
1mo
Save
Mark Applied
Hide
Evaluations Engineering - Member of Technical Staff
San Francisco, California, United States
$200k-$400k/yr OnsiteFull Time
Simile
Simile: Building AI simulations to predict human behavior at scale.
Several years building production backend and data systems, experience with evaluation pipelines, ML/LLM fluency, strong system design and communication skills.
Job PostingView all
1mo
Save
Mark Applied
Hide
Research Engineer - Evaluations
Redwood City, California, United States
HybridFull Time
Luma AI
Luma AI: Develops multimodal AI for video generation and creative production.
5+ YOE5+ years building ML evaluation systems or large-scale model pipelines; master's/PhD or equivalent; experience with visual data; proficiency in Python and PyTorch/JAX/TensorFlow; strong software engineering and distributed systems skills.
Python, PyTorch, JAX, TensorFlow, CI/CD
Job PostingView all
4w
Save
Mark Applied
Hide
Member of Technical Staff - Research Software Engineer - Safety Evaluations Infrastructure
New York City or San Francisco or London
OnsiteFull Time
Reflection AI
Reflection AI: Developing open-source frontier artificial intelligence models and coding agents.
Experience building secure, isolated infrastructure for sensitive model evaluations; strong Python skills; security engineering fundamentals; data pipeline and sandboxing experience.
Python
Job PostingView all
3mo
Save
Mark Applied
Hide
Staff Software Engineer, Quantitative Evaluations
Mountain View or San Francisco
$251k-$310k/yr HybridFull Time
Waymo
Waymo: Autonomous driving technology for ride-hailing and logistics.
4+ YOEBS in a quantitative field; 4-7 years experience; strong C++, Python, SQL skills; data analysis; building data pipelines; ML familiarity; strong coding standards.
C++, Python, SQL, Machine Learning, Data Processing, Statistics
Job PostingView all
3mo
Save
Mark Applied
Hide
Sr. Condition Evaluation Engineer
Long Beach, California, United States
OnsiteFull Time
Twining Consulting
Twining Consulting: Provides construction materials testing, inspection, and geotechnical engineering services.
Senior engineer with a degree in civil/structural engineering and experience in condition surveys, nondestructive evaluation, and as-built documentation.
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
Santa Clara, California, United States
$201k-$352k/yr OnsiteFull Time
ServiceNow
ServiceNowNYSE: NOW: Provides a cloud platform for automating enterprise digital workflows.
8+ YOE3+ Mgmt8+ years software/ML engineering experience with 3+ years managing AI/ML teams; expertise in LLM evaluation, benchmarking, Python, SQL, Pandas, NumPy, and ML evaluation metrics.
Python, SQL, Pandas, NumPy, ROUGE, BLEU, G-Eval, OpenAI, Anthropic, Google
Job PostingView all
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
Yes Energy
Yes Energy: Provides real-time electric power market data and trading analytics.
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
Job PostingView all
2mo
Save
Mark Applied
Hide
AI Engineer, Evaluation
San Francisco or New York City
$150k-$250k/yr HybridFull Time
Distyl AI
Distyl AI: Builds AI-native orchestration platforms for enterprise operations.
2+ YOE2+ years software engineering, strong Python, experience with evaluation- or experiment-driven development, ability to encode human judgment into tests/graders, systems-oriented mindset, and willingness to travel 10–50%.
Python, LLM
Job PostingView all
1w
Save
Mark Applied
Hide
Electrical Engineer
Sunnyvale, California, United States
$74k-$116k/yr OnsiteFull Time
Abbott
AbbottNYSE: ABT: Manufactures medical devices, diagnostics, and nutritional health products.
3+ YOEBachelor's degree in electrical, electronic, or related engineering; 3+ years designing and evaluating PCBs, fixtures, and tools; experience with testing, documentation, FDA requirements, and engineering laboratory equipment.
Printed Circuit Boards (PCBs), oscilloscopes, spectrum analyzers, arbitrary waveform generators, function generators, LCR meters, power supplies, embedded systems, DSP, SPICE, Matlab, COMSOL, spreadsheet, word processing, database management
Job PostingView all
2w
Save
Mark Applied
Hide
Founding Research Engineer
New York City or San Francisco
$215k-$330k/yr OnsiteFull Time
Ambral
Ambral: AI-powered account managers for enterprise customer success teams.
4+ YOE4+ years building production software or ML systems, including 2+ years with RL environments or evaluation/agent infrastructure; strong software engineering, data processing, reproducibility, and evaluation skills.
Job PostingView all
3mo
Save
Mark Applied
Hide
Senior Engineer, Design Evaluation Engineering
San Jose, California, United States
$157k-$188k/yr OnsiteFull Time
Analog Devices
Analog DevicesNASDAQ: ADI: Designs and manufactures semiconductors for signal processing and power management.
1+ YOEMaster’s degree in Electrical Engineering and 1 year of relevant design evaluation experience; proficient in C#/C++, Python, and PCB design tools; able to automate tests and analyze results.
C#, C++, Python, Visual Studio, OrCAD, Cadence Allegro, Microsoft Office, PCB schematic capture, PCB layout
Job PostingView all
1mo
Save
Mark Applied
Hide
Materials Engineer II
Torrance, California, United States
$115k-$150k/yr OnsiteFull Time
Antares
Antares: Designing and manufacturing microreactors for defense and space missions.
2+ YOEBachelor's in materials/mechanical engineering, 2+ years materials engineering experience, proficiency in optical/electron microscopy and mechanical property evaluation, materials characterization and failure analysis.
optical microscopy, electron microscopy
Job PostingView all
1w
Save
Mark Applied
Hide
AI Engineer – Algorithm Evaluation & Agentic Systems
Sunnyvale, California, United States
OnsiteFull Time
Apple
AppleNASDAQ: AAPL: Designs and sells consumer electronics, software, and online services.
Experience evaluating AI algorithms and designing agentic systems, with system-level reasoning, metric definition, engineering rigor, and clear technical communication.
Job PostingView all
1d
Save
Mark Applied
Hide
Lead Test and Evaluation Engineer
El Segundo, California, United States
$135k-$224k/yr OnsiteFull Time
Boeing
BoeingNYSE: BA: Designs and manufactures aircraft, defense systems, and space technology.
3+ YOEBachelor's degree in engineering or a related technical field, 3+ years in complex-systems verification and validation and test evaluation engineering; 5+ years leading complex T&E projects preferred.
DFMAT, Test & Special Test Equipment (STE), CONOPS, Program Engineering Review Board (ERB), Database Change Request (DCR), SITP, STVP, Test Requirements Documents (TRD), PCAB, CAPA
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
Santa Clara, California, United States
$201k-$352k/yr OnsiteFull Time
ServiceNow
ServiceNowNYSE: NOW: Enterprise cloud platform for digital workflow automation.
8+ YOE3+ MgmtRequires 8+ years of software engineering or machine learning experience, 3+ years leading AI/ML teams, Python, SQL, Pandas, NumPy, AI evaluation metrics, MLOps platforms, and a bachelor's degree in a quantitative field.
OpenAI, Anthropic, Google, ROUGE, BLEU, G-Eval, Python, SQL, Pandas, NumPy, MLOps, Knowledge Graphs, CMDBs, RAG
Job PostingView all
1mo
Save
Mark Applied
Hide
Evaluation and ML Systems Engineer, AI Safety and Security Engineering
Santa Clara or North Carolina or Tennessee or New York or California or Florida
$152k-$242k/yr HybridFull Time
NVIDIA
NVIDIANASDAQ: NVDA: Designs graphics processing units and artificial intelligence hardware.
5+ YOE5+ years ML engineering or evaluation experience, bachelor's degree or equivalent, benchmark and metrics design, Python engineering, experiment tracking, and statistical evaluation skills.
Python
Job PostingView all
1w
Save
Mark Applied
Hide
Electrical Engineer
Sunnyvale, California, United States
$74k-$116k/yr OnsiteFull Time
Abbott
AbbottNYSE: ABT: Provides medical devices, diagnostics, and science-based nutritional products.
3+ YOEBachelor's degree in electrical, electronic, or related engineering and 3+ years of PCB, fixture, and tool construction and evaluation experience. Requires FDA compliance, testing, documentation, communication, and analytical skills.
Printed Circuit Boards (PCBs), oscilloscopes, spectrum analyzers, arbitrary waveform generators, function generators, LCR meters, power supplies, embedded systems, DSP, SPICE, Matlab, COMSOL
Job PostingView all

Explore Jobs

Expand Your Job Search