23 ai evaluation engineer jobs at 20 companies in Los Angeles, CA
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
1mo
Save
Mark Applied
Hide
AI Engineer, Agents and Evaluation
Torrance, California, United States
OnsiteFull Time
Proven production experience shipping agent or tool-use systems, experience self-hosting open-weights models on GPUs, strong orchestration/serving/evaluation skills, and habit of treating hallucination as engineering problem.
vLLM, TGI
2w
Save
Mark Applied
Hide
Staff AI Platform Engineer
San Francisco or Los Angeles or Denver or Austin or Chicago or New York or Seattle or Toronto or Santa Barbara or San Diego
$180k-$273k/yr RemoteFull Time
7+ YOERequires 7+ years in AI/ML platform engineering, strong Python and/or TypeScript, distributed systems, APIs, evaluation, observability, and AI infrastructure experience; bachelor's degree or equivalent practical experience.
Python, TypeScript, JavaScript, React, Kubernetes, Docker, Terraform, LangChain, LangGraph, LlamaIndex, LiveKit, REST, gRPC, MCP, A2A
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1w
Save
Mark Applied
Hide
Applied AI Engineer
San Francisco or Los Angeles or Washington or San Diego or Seattle or London
$150k-$170k/yr HybridFull Time
2+ YOEBachelor's or master's degree in a technical field, 2+ years of software development, strong Python, production systems, AI/ML integration, model evaluation, and familiarity with APIs, distributed systems, containers, CI/CD, observability, and edge deployment.
Python, APIs, CI/CD, GPU
2mo
Save
Mark Applied
Hide
Senior AI Engineer
United States or New York City or Palm Beach or Los Angeles or Miami or Greenwich or Westchester County or The Hamptons or San Francisco or Manhattan
$190k-$250k/yr RemoteFull Time
4+ YOE4+ years building and operating LLM-powered production ML systems, strong Python, backend framework experience (FastAPI/Flask), LLM orchestration and retrieval design, Snowflake/dbt/Dagster/Airflow familiarity, evaluation systems, HIPAA/PHI knowledge, and strong written communication.
Python, FastAPI, Flask, Snowflake, dbt, Dagster, Airflow, vLLM, Triton, TGI, SGLang, LOINC, SNOMED CT, ICD-10, RxNorm, FHIR, HL7
2mo
Save
Mark Applied
Hide
AI Engineering Lead
Los Angeles, California, United States
$225k-$250k/yr HybridFull Time
8+ YOE8+ years software engineering experience, strong Python skills, experience leading enterprise AI initiatives, LLM fundamentals, AWS Bedrock, Anthropic Claude, multi-agent frameworks, model evaluation, governance, and stakeholder influence.
Python, AWS Bedrock, Anthropic Claude, Claude Code, Langfuse, LangGraph, LangChain, MCP, Snowflake, Oracle
4w
Save
Mark Applied
Hide
Senior AI/ML Engineer - Agentic AI
Newport Beach, California, United States
$131k-$272k/yr HybridFull Time
6+ YOEBS/MS/PhD in CS or related field; 6+ years shipping ML systems with 3+ years hands-on LLM/agent production experience; expert Python, PyTorch, inference optimization, agent frameworks, evaluation and observability.
Python, PyTorch, LoRA, QLoRA, PEFT, vLLM, TensorRT-LLM, int8, int4, GPTQ, AWQ, LangGraph, LangChain, CrewAI, AutoGen, MCP, Claude Code, OpenCode, Langfuse, LangSmith, Arize, SFT, DPO, RLHF
2mo
Save
Mark Applied
Hide
Sr. AI Agent / Prompt Engineer
Irvine, California, United States
HybridFull Time
8+ YOEDesign and deploy production AI agents using LLMs, build prompt systems and tool orchestration, evaluate agent behavior, integrate with enterprise systems; strong Python and agent-framework experience required.
Python, LangChain, LangGraph
3w
Save
Mark Applied
Hide
Application Software Engineer, Applied AI
Bastrop or Cape Canaveral or Hawthorne or McGregor or Redmond or Brownsville or Vandenberg
OnsiteFull Time
1+ YOEDesign, evaluate, deploy, and integrate AI systems including LLMs; 1+ years full-stack experience; bachelor's in CS/engineering/math or 2+ years professional software experience; strong software engineering and AI production experience.
Python, C#.NET, Go, Scala, Java, Docker, Kubernetes, Kafka, Angular, React, PostgreSQL, SQL Server, LLM
2mo
Save
Mark Applied
Hide
Information Technology - Principal Platform Engineer, Enterprise AI Enablement
Vancouver or Seattle or Toronto or New York City or Los Angeles
OnsiteFull Time
Deep enterprise AI and platform architecture experience, LLMOps/MLOps, AI guardrails and observability, vector DBs and data orchestration, vendor evaluation, and cross-functional leadership for enterprise AI enablement.
Azure AI Studio, AWS Bedrock, Google Cloud Vertex AI, LLMOps, MLOps, RAG
2mo
Save
Mark Applied
Hide
Principal, Applied AI Enablement, Legal
West Hollywood, California, United States
$150k-$220k/yr OnsiteFull Time
Experienced AI/product builder to prototype, ship, and iterate AI tools for legal work; strong applied ML/AI judgment, prototyping, evaluation, and ability to partner with legal, engineering, and ops teams.
1d
Save
Mark Applied
Hide
Senior Product Manager, AI Safety Evaluation & Governance - TikTok Safety Product
San Jose or Los Angeles or Singapore or New York City or London or Dublin or Paris or Berlin or Dubai or Jakarta or Seoul or Tokyo
$186k-$374k/yr OnsiteFull Time
5+ YOEBachelor's degree in a quantitative field and 5+ years in AI/ML evaluation, model quality, trust and safety, or data-driven product management; SQL, LLM, prompt engineering, and Chinese fluency required.
SQL, Large Language Models (LLMs)
1d
Save
Mark Applied
Hide
Senior AI Applied Manager
San Francisco or Chicago or Denver or Los Angeles or Phoenix
$125k-$186k/yr OnsiteFull Time
8+ YOERequires 8+ years of software engineering experience, expert Python, strong AWS and data skills, LLM evaluation and agentic systems experience, communication ability, and a bachelor's degree in computer science or engineering.
Python, AWS, SQL, Dataiku (DSS), OpenAI AgentKit, ChatKit, Apps SDK/MCP
2mo
Save
Mark Applied
Hide
Space-to-Ground Systems Integration Engineer
El Segundo or Chantilly or Colorado Springs
$100k-$176k/yr OnsiteFull Time
5+ YOEBachelor's in engineering/physics/computer science (or related); 5+ years Test & Evaluation and space systems integration experience (8+ for specialist); secret clearance required (active preferred or ability to obtain); experience with requirements flow-down, SIP/SEP/STIP, test planning/execution, cloud (AWS, Microsoft Azure, GCP), systems enginee...
Cameo, STK, Simulink, Python, MATLAB, AWS, Microsoft Azure, GCP, CI/CD, DevSecOps, AI/ML
2w
Save
Mark Applied
Hide
Applied LLM Systems Engineer
Costa Mesa or Santa Ana
$112k-$149k/yr OnsiteFull Time
5+ YOERequires 5+ years of software engineering, production AI delivery, Python proficiency, LLM optimization, evaluation and observability experience, and eligibility for Secret or higher US security clearance.
Python, Lattice OS, S1000D, DITA, MIL-STD-40051, CI/CD
1mo
Save
Mark Applied
Hide
Senior Software Engineer, Legal — CoCounsel Forward Deployed Engineering
Frisco or New York City or Toronto or Ann Arbor or Eagan or San Francisco or Los Angeles or Irvine or McLean or Washington
$95k-$176k/yr HybridFull Time
5+ YOEJD or formal legal training (or equivalent experience),5+ years software engineering,proficiency in Python,AI/LLM evaluation and prompt engineering,ability to engage legal users and mentor engineers,experience delivering production-grade SaaS solutions.
Python
1mo
Save
Mark Applied
Hide
Lead Machine Learning Engineer
San Francisco or Seattle or Santa Monica
$188k-$252k/yr HybridFull Time
7+ YOE7+ years building and deploying ML models (recommendation systems preferred), strong ML/AI/LLM experience, proficiency with PyTorch, TensorFlow, Databricks, Spark, SQL, strong communication and evaluation skills.
PyTorch, TensorFlow, Databricks, Spark, SQL
3mo
Save
Mark Applied
Hide
Legal Engineering Attorney
New York or Boston or Los Angeles or Chicago or Washington
$220k-$250k/yr HybridFull Time
3+ YOEJ.D. required; 3+ years substantive legal practice and 2+ years legal engineering experience. Experience authoring prompts and working with JSON/JSON Schema/YAML/Markdown and evaluating AI outputs.
JSON, JSON Schema, YAML, Markdown
2mo
Save
Mark Applied
Hide
Senior Manager, Enterprise Data
El Segundo, California, United States
$171k-$233k/yr OnsiteFull Time
7+ YOE7+ years software/data engineering experience with hands-on AI/ML; data platform architecture (Databricks, Snowflake), vendor evaluation for AI, RAG experience, security compliance (ITAR/EAR), technical leadership and hands-on coding.
Databricks, Fivetran, dbt, Snowflake, Airbyte, Looker, Tableau, Power BI, Databricks dashboards, ION-Ramp
Explore Jobs
Expand Your Job Search
Top Cities Near Los Angeles for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Long Beach, CA
- AI Evaluation Engineer jobs in Anaheim, CA
- AI Evaluation Engineer jobs in Santa Ana, CA
- AI Evaluation Engineer jobs in Riverside, CA
- AI Evaluation Engineer jobs in Irvine, CA
- AI Evaluation Engineer jobs in Fontana, CA
- AI Evaluation Engineer jobs in Huntington Beach, CA
- AI Evaluation Engineer jobs in Glendale, CA
- AI Evaluation Engineer jobs in Santa Clarita, CA
- AI Evaluation Engineer jobs in Garden Grove, CA
- AI Evaluation Engineer jobs in Rancho Cucamonga, CA
- AI Evaluation Engineer jobs in Ontario, CA
- AI Evaluation Engineer jobs in Corona, CA
- AI Evaluation Engineer jobs in Lancaster, CA
- AI Evaluation Engineer jobs in Palmdale, CA
- AI Evaluation Engineer jobs in Pomona, CA
- AI Evaluation Engineer jobs in Torrance, CA
- AI Evaluation Engineer jobs in Pasadena, CA
- AI Evaluation Engineer jobs in Orange, CA
- AI Evaluation Engineer jobs in Fullerton, CA