HiringCafe
Join our community

238 ai evaluation engineer jobs at 136 companies in Berkeley, CA

2mo
Save
Mark Applied
Hide
AI Engineer, Evaluation
San Francisco or New York City
$150k-$250k/yr HybridFull Time
Distyl AI
Distyl AI: Builds AI-native orchestration platforms for enterprise operations.
2+ YOE2+ years software engineering, strong Python, experience with evaluation- or experiment-driven development, ability to encode human judgment into tests/graders, systems-oriented mindset, and willingness to travel 10–50%.
Python, LLM
Job PostingView all
2w
Save
Mark Applied
Hide
AI Engineer – Algorithm Evaluation & Agentic Systems
Sunnyvale, California, United States
OnsiteFull Time
Apple
AppleNASDAQ: AAPL: Designs and sells consumer electronics, software, and online services.
Experience evaluating AI algorithms and designing agentic systems, with system-level reasoning, metric definition, engineering rigor, and clear technical communication.
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Director - AI Evaluation Platform
Santa Clara, California, United States
$279k-$488k/yr HybridFull Time
ServiceNow
ServiceNowNYSE: NOW: Provides a cloud platform for automating enterprise digital workflows.
Proven leadership of large AI or platform engineering teams, deep expertise in AI evaluation methodologies, production evaluation platforms, strong communication, and hands-on experience with PyTorch or TensorFlow.
PyTorch, TensorFlow
Job PostingView all
1w
Save
Mark Applied
Hide
Sr. Evaluation Engineer
San Francisco, California, United States
$158k-$218k/yr HybridFull Time
LogicMonitor
LogicMonitor: Provides AI-powered hybrid observability for enterprise IT infrastructure.
5+ YOE5+ years in software engineering, machine learning, or applied AI; strong Python skills; experience with AI evaluation, LLMs, agents, retrieval-augmented generation, testing, monitoring, and CI/CD.
Python, LangSmith, Arize Phoenix, Braintrust, DeepEval, Ragas, TruLens, OpenAI Evals, MLflow, CI/CD
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
Capital One
Capital OneNYSE: COF: Provides credit card, banking, and auto loan services.
4+ YOEBachelor's plus 6 years or master's plus 4 years in AI/ML development; 6 years programming in Python, Go, Scala, or Java; experience building scalable AI systems and leading engineering teams.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, Google Cloud, Azure, C++, C#, Golang
Job PostingView all
1w
Save
Mark Applied
Hide
Software Engineer, AI Evaluation
San Francisco, California, United States
$148k-$232k/yr HybridFull Time
Nuna
Nuna: Provides data analytics and software for value-based healthcare solutions.
Significant production systems experience; expertise evaluating AI systems, agentic tooling, adversarial testing, statistics, experimental design, and functional UI development for non-engineers.
LangSmith, Braintrust, DeepEval, Ragas, Promptfoo
Job PostingView all
1w
Save
Mark Applied
Hide
Applied AI Engineer
San Francisco, California, United States
OnsiteFull Time
Magic Patterns
Magic Patterns: AI-powered platform for rapid software prototyping and design.
Experience using AI tools, working with AI models or agents, and building evaluation systems, post-training, or context engineering; strong first-principles problem-solving skills.
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior AI Engineer — Agents
San Francisco, California, United States
HybridFull Time
Mira Mace
Mira Mace: AI-powered healthcare advocacy and navigation for Medicare beneficiaries.
Experience building and shipping LLM-powered systems or agents, strong AI fluency (prompting, retrieval, evaluation), production monitoring, and solid software engineering practices.
Python, TypeScript, FastAPI, Next.js, Postgres
Job PostingView all
1mo
Save
Mark Applied
Hide
Applied AI Engineer
San Francisco, California, United States
HybridFull Time
Abby Care
Abby Care: Trains and pays family members to provide home care.
2+ YOE2+ years professional software/ML or applied AI experience, strong software engineering, experience with language models/AI APIs, backend and data pipeline skills, ability to evaluate models and work with healthcare workflows.
Python
Job PostingView all
2w
Save
Mark Applied
Hide
AI Engineer
Mountain View, California, United States
OnsiteFull Time
AI Fund
AI Fund: Builds and invests in artificial intelligence startups.
3+ YOERequires 3+ years of software engineering experience, production LLM systems experience, strong Python and/or TypeScript/Node skills, and experience with LLM APIs, agentic workflows, evaluation, and end-to-end services.
Claude, OpenAI, Python, TypeScript, Node
Job PostingView all
2mo
Save
Mark Applied
Hide
Applied AI Engineer
San Francisco, California, United States
$171k-$242k/yr HybridFull Time
Artos
Artos: AI-powered document authoring platform for life sciences R&D.
2+ YOE2+ years building and deploying AI/ML applications, hands-on LLM experience, backend engineering with Python, API development (FastAPI/Django), cloud container deployment, R&D on model capabilities, and evaluation tooling experience.
Langfuse, LangSmith, FastAPI, Django, AWS, GCP, Azure, Terraform, Pulumi, GitHub Actions, React, Python, OpenAI, Anthropic, Google
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or New York City or San Jose or Cambridge
$230k-$286k/yr OnsiteFull Time
Capital One
Capital OneNYSE: COF: A diversified financial services providing banking and credit products.
6+ YOEBachelor's degree plus 6 years or master's degree plus 4 years in AI/ML; 6 years programming with Python, Go, Scala, or Java; cloud AI deployment and team leadership preferred.
AWS Ultraclusters, Hugging Face, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, AWS, Google Cloud, Azure, C++, C#, Golang
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Lead AI Engineer (SDK's: Gen AI Evaluation and MCP)
McLean or San Francisco or Cambridge or San Jose or New York City
$230k-$286k/yr OnsiteFull Time
Capital One
Capital OneNYSE: COF: Financial services offering credit cards, banking, and loans.
6+ YOEBachelor's plus 6+ years (or Master's plus 4+ years) developing AI/ML systems; strong programming (Python, Go, Scala, Java); cloud AI deployment experience; leadership and research application skills.
AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, Python, Go, Scala, Java, C++, C#, Golang, AWS, Google Cloud, Azure
Job PostingView all
4w
Save
Mark Applied
Hide
Founding Engineer (Applied AI)
San Francisco, California, United States
OnsiteFull Time
Polymr
Polymr: AI-native procurement and manufacturing workflow automation platform.
Experience building document ingestion/extraction, agentic workflows, and evaluation harnesses; familiarity with AI coding tools and working closely with customers.
Claude Code, Cursor
Job PostingView all
1w
Save
Mark Applied
Hide
Applied AI Engineer
Canada or San Francisco or Raleigh or Boston
HybridFull Time
MaintainX
MaintainX: Provides mobile-first software for maintenance and asset management.
Applied GenAI experience with prompt engineering, structured outputs, RAG, evaluation datasets, production LLM features, multimodal inputs, and model quality, cost, and latency optimization.
LLMX, Attachments, OCR, RAG
Job PostingView all
3mo
Save
Mark Applied
Hide
Lead AI Engineer — Productivity Systems
Durham or Miami or Palo Alto or Washington
$12k-$18k/mo HybridFull Time
Nubank
NubankNYSE: NU: Digital financial platform offering banking, credit, and investment services.
Proven experience shipping LLM-powered systems in production; strong prompt/context engineering, evaluation and guardrail development, and software engineering skills (Python/TypeScript/Clojure).
Python, TypeScript, Clojure, n8n, Zapier, AWS, Claude Code, Cursor, Copilot, Slack, Google Workspace, Jira, Terraform, RAG, MCP
Job PostingView all
3mo
Save
Mark Applied
Hide
AI Engineer
San Francisco, California, United States
$180k-$250k/yr OnsiteFull Time
Fluency
Fluency: Automates enterprise process mapping and documentation with artificial intelligence.
Hands-on experience building LLM-powered features; production engineering in TypeScript/Node and/or Python; experience with multiple LLM providers; strong prompt engineering, retrieval and evaluation; track record of evaluation systems; solid software engineering.
TypeScript, Node.js, Python, AWS Lambda, ECS Fargate, OpenAI, Anthropic, Google Vertex, AWS Bedrock, pgvector
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Software Engineer, AI Infrastructure - LVM Inference & Evaluation
Redwood City, California, United States
$168k-$205k/yr HybridFull Time
Ambient.ai
Ambient.ai: AI-powered physical security platform for proactive threat detection.
4+ YOE4+ years building infrastructure or production AI systems; strong Python; experience with ML infrastructure, LLM/LVM inference, inference optimization, evaluation frameworks, cloud and GPU workloads; BS/MS or equivalent.
Python, vLLM, Triton Inference Server, CUDA, NCCL, PyTorch, TensorRT, ONNX
Job PostingView all
2d
Save
Mark Applied
Hide
AI Engineer
Waterloo or Waterloo or Kitchener or Toronto or New York City or San Francisco
HybridFull Time
Cambio
Cambio: AI-native software for commercial real estate decarbonization and operations.
Experience with Python web development; shipped AI features; hands-on LLMs or generative AI; evaluation frameworks; vector databases; retrieval architectures; and model cost, latency, and accuracy tradeoffs.
Python, Django, LLMs, Retrieval-Augmented Generation (RAG), vector databases
Job PostingView all
1w
Save
Mark Applied
Hide
AI Support Engineer
San Jose or Austin
$96k-$148k/yr OnsiteFull Time
Advantest
AdvantestTokyo Stock Exchange: 6857: Manufacturer of automated test systems for the semiconductor industry.
Bachelor's degree in engineering, computer science, information systems, or equivalent; experience evaluating technical solutions; AI familiarity; analytical, troubleshooting, communication, and project leadership skills.
LLMs, RAG, AI agents, MCP
Job PostingView all

Explore Jobs

Expand Your Job Search