34 ai evaluation engineer jobs at 25 companies in Illinois
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
3mo
Save
Mark Applied
Hide
AI Engineer
Plano or Purchase or Chicago
$94k-$156k/yr OnsiteFull Time
6+ YOEBachelor in CS/AI/ML or equivalent; 6-8 years software lifecycle; ML/LLM/agent systems; Python/Java; production-grade AI delivery; strong evaluation and automation.
Python, Java, FastAPI, asyncio, Pydantic, LangChain, LangGraph, CrewAI, MLflow, Arise, Docker, Kubernetes, Ray, Kafka, gRPC, Redis, Prometheus, OpenTelemetry, Grafana
2w
Save
Mark Applied
Hide
Staff AI Platform Engineer
San Francisco or Los Angeles or Denver or Austin or Chicago or New York or Seattle or Toronto or Santa Barbara or San Diego
$180k-$273k/yr RemoteFull Time
7+ YOERequires 7+ years in AI/ML platform engineering, strong Python and/or TypeScript, distributed systems, APIs, evaluation, observability, and AI infrastructure experience; bachelor's degree or equivalent practical experience.
Python, TypeScript, JavaScript, React, Kubernetes, Docker, Terraform, LangChain, LangGraph, LlamaIndex, LiveKit, REST, gRPC, MCP, A2A
4w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1mo
Save
Mark Applied
Hide
Senior AI Platform Engineer
San Francisco or Alabama or Arkansas or California or Colorado or Florida or Georgia or Illinois or Kentucky or Massachusetts or Michigan or Montana or Missouri or Nevada or New Jersey or New York or North Carolina or Oregon or Pennsylvania or Texas or Washington or Utah or Virginia or Wisconsin
$156k-$231k/yr HybridFull Time
3+ YOE3+ years building production software or ML/data systems; strong Python and systems/API design; experience with LLM retrieval/RAG, agents, evaluation, Databricks/MLflow and modern data stacks.
Python, PySpark, dbt, Databricks, Delta Lake, Unity Catalog, MLflow, Astronomer, Airflow, Claude, GitHub, Shortcut, Notion, pgvector, Pinecone, Weaviate, LangGraph, MCP
1mo
Save
Mark Applied
Hide
Senior AI Engineer
Chicago or Hong Kong or London or New York City or Singapore
$200k-$300k/yr OnsiteFull Time
5+ YOE5+ years software engineering with strong Python; production fine-tuning/distillation of open-weight models; on-prem LLM serving, GPU infrastructure and Kubernetes experience; model evaluation and tooling.
Python, Llama, Qwen, Mistral, vLLM, TGI, Triton, Kubernetes, Hugging Face
3d
Save
Mark Applied
Hide
Oliver Wyman - Senior Lead Engineer
New York City or Boston or Washington or Chicago
$195k-$250k/yr HybridFull Time
Requires a quantitative or computational science background, large-scale production AI systems experience, modern AI programming fluency, agent platform knowledge, LLM design expertise, AI evaluation and governance skills, and strong mathematical grounding.
Azure Foundry, AWS Bedrock, Gemini Enterprise Agent Platform, Open-Source
1d
Save
Mark Applied
Hide
Remote Software Developer – AI Research & Evaluation
United States or Chicago or San Francisco
$200-$300/hr RemoteContract
3+ YOE3+ years of software engineering experience with strong systems, infrastructure, or backend expertise; production-scale development, architecture, debugging, code review, and excellent written and oral communication required.
Python, C/C++, Rust, Go, Java, JavaScript, ReactJS
3d
Save
Mark Applied
Hide
Oliver Wyman - Senior Lead Engineer
New York City or Boston or Chicago or Washington
$195k-$250k/yr HybridFull Time
Requires a quantitative or computational technical background and proven experience designing and deploying large-scale AI systems, agent platforms, production pipelines, evaluation frameworks, and LLM solutions.
Azure Foundry, AWS Bedrock, Gemini Enterprise Agent Platform
1w
Save
Mark Applied
Hide
Sr ML/AI Engineer
Arizona or Georgia or Idaho or Illinois or Florida or Massachusetts or Maine or Michigan or Missouri or North Carolina or New York or Pennsylvania or Tennessee or Texas or Oregon or Virginia or Washington or Durham
RemoteFull Time
Bachelor’s degree in a quantitative field, significant hands-on ML experience, advanced machine learning and applied statistics expertise, strong Python skills, and experience productionizing, monitoring, and evaluating models.
Python, Snowflake, Snowpark ML, Snowflake Feature Store, Snowflake Model Registry, AWS, LLM, retrieval-augmented generation
4d
Save
Mark Applied
Hide
Operations Research & Applied AI Engineer
Downers Grove, Illinois, United States
$100k-$120k/yr HybridFull Time
5+ YOEBachelor's or equivalent experience, 5+ years applying mathematical optimization to production problems, strong mixed-integer programming, Python and SQL, AI evaluation, and production LLM application experience.
SCIP, Gurobi, CPLEX, Python, SQL, Power BI, DAX, SSAS, SSIS, Azure OpenAI
3w
Save
Mark Applied
Hide
Senior Software Engineer, AI/ML
Chicago, Illinois, United States
$72k-$141k/yr HybridFull Time
4+ YOERequires a bachelor's degree or equivalent experience, 4+ years of application development, advanced Python, AI/ML, LLMs, Vertex AI, agent frameworks, prompt engineering, and model evaluation experience.
Python, Google Vertex AI, Gemini, OpenAI, Google Agent Development Kit, LangChain, LangGraph, LangSmith, LangFuse, Phoenix, Arize, Eval AI Testing
4d
Save
Mark Applied
Hide
Operations Research & Applied AI Engineer
Downers Grove, Illinois, United States
$100k-$120k/yr HybridFull Time
5+ YOEBachelor's degree or equivalent experience, 5+ years applying mathematical optimization to production problems, strong Python, SQL, mixed-integer programming, solver, AI evaluation, communication, and analytical skills.
SCIP, Gurobi, CPLEX, Python, SQL, Power BI, DAX, SSAS, SSIS, Azure OpenAI
1mo
Save
Mark Applied
Hide
Agentic AI Architect - Intelligence Engineering (Central)
Austin or Chicago or Columbus or Dallas or Detroit or Houston or Indianapolis or Kansas City or Minneapolis or Nashville or St. Louis
$108k-$200k/yr HybridFull Time
5+ YOE5+ years software engineering experience with 2+ years focused on generative/agentic AI; experience designing multi-agent systems, RAG, cloud AI platforms, Python, API frameworks, evaluation/observability, and leading small teams.
Amazon Web Services, Azure, Google Cloud, AWS Bedrock, Azure AI Foundry, OpenAI Service, GCP Vertex AI, Gemini, Databricks, Python, FastAPI, Flask, Lang Graph, Strands Agents SDK, OpenAI Agents SDK, Google ADK, LangSmith, RAGAS, DeepEval, Claude Code, Cursor, Kiro, AutoGen, CrewAI, Semantic Kernel, Model Context Protocol (MCP)
4d
Save
Mark Applied
Hide
Operations Research & Applied AI Engineer
Downers Grove, Illinois, United States
$100k-$120k/yr HybridFull Time
5+ YOEBachelor's degree or equivalent experience, 5+ years applying mathematical optimization to production problems, mixed-integer programming, solver experience, Python, SQL, production AI, evaluation methodology, and strong communication skills.
SCIP, Gurobi, CPLEX, Python, SQL, Power BI, DAX, SSAS, SSIS, Azure OpenAI
3w
Save
Mark Applied
Hide
Manager, Conversational AI
United States or California or Florida or Georgia or Idaho or Illinois or Massachusetts or Michigan or New Jersey or New York or North Carolina or Ohio or Oregon or Pennsylvania or Texas or Virginia or Washington or Wisconsin
RemoteFull Time
Experience leading teams and building conversational AI products, including LLM-powered agents, evaluation frameworks, optimization, experimentation, and cross-functional delivery with Product, Engineering, Data, and AI teams.
AI, Machine Learning, LLM
1w
Save
Mark Applied
Hide
Senior Software Engineer - Agentic AI Engineer
Dallas or Atlanta or Chicago or Nashville
$73k-$174k/yr HybridFull Time
10+ YOE10+ years of related experience; advanced SQL and Python skills; experience with Vertex AI, Model Garden, Vertex AI Pipelines, model evaluation, BigQuery, Google Cloud Storage, and real-time data processing.
Vertex AI Agent Builder, Agent Developer Kit, Model Context Protocol (MCP) Toolbox, BigQuery, Spanner, Vertex AI, Dataflow, RunInference API, AlloyDB, Model Garden, Vertex AI Pipelines, SQL, Python, Google Cloud Storage
3w
Save
Mark Applied
Hide
Deployed Engineer (Chicago)
Chicago, Illinois, United States
$150k-$250k/yr RemoteFull Time
6+ YOERequires 6+ years in a relevant technical role, strong Python and JavaScript, systems fundamentals, production AI agent experience, customer-facing technical evaluation skills, and clear communication.
Python, JavaScript, LangChain, LangGraph, AWS, GCP, Azure, Kubernetes
2mo
Save
Mark Applied
Hide
Senior Applied AI Solutions Architect — Amazon Connect, Applied AI SA - AIVT
Seattle or Arlington or Atlanta or New York or Denver or Herndon or Chicago or Mountain View or San Francisco
$154k-$239k/yr OnsiteFull Time
7+ YOE7+ years in technology domains; 3+ years in design/implementation; familiarity with MCP/A2A; AI/ML concepts incl. LLMs, prompt engineering, RAG, model evaluation
Python, Node.js, AWS Lambda, API Gateway, Step Functions, Bedrock, MCP, RAG, DynamoDB, RDS, S3, OpenSearch, Kendra
1d
Save
Mark Applied
Hide
AI Systems Architect
New York City or Chicago or Boston or San Francisco or Palo Alto or Toronto
$130k-$170k/yr RemoteFull Time
3+ YOE3+ years in product or forward-deployed engineering, experience turning ambiguous problems into production systems, and hands-on experience deploying LLM systems with evaluations, latency, cost, regressions, and failure handling.
OpenAI, ChatGPT, LLMs
Explore Jobs
Expand Your Job Search
Top Cities in Illinois for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Chicago, IL
- AI Evaluation Engineer jobs in Aurora, IL
- AI Evaluation Engineer jobs in Rockford, IL
- AI Evaluation Engineer jobs in Joliet, IL
- AI Evaluation Engineer jobs in Naperville, IL
- AI Evaluation Engineer jobs in Springfield, IL
- AI Evaluation Engineer jobs in Peoria, IL
- AI Evaluation Engineer jobs in Elgin, IL
- AI Evaluation Engineer jobs in Waukegan, IL
- AI Evaluation Engineer jobs in Champaign, IL
- AI Evaluation Engineer jobs in Cicero, IL
- AI Evaluation Engineer jobs in Bloomington, IL
- AI Evaluation Engineer jobs in Arlington Heights, IL
- AI Evaluation Engineer jobs in Evanston, IL
- AI Evaluation Engineer jobs in Schaumburg, IL
- AI Evaluation Engineer jobs in Bolingbrook, IL
- AI Evaluation Engineer jobs in Decatur, IL
- AI Evaluation Engineer jobs in Palatine, IL
- AI Evaluation Engineer jobs in Skokie, IL
- AI Evaluation Engineer jobs in Des Plaines, IL