26 ai evaluation engineer jobs at 18 companies in Colorado
2mo
Save
Mark Applied
Hide
AI Evaluation Engineer
Bucharest or Boulder or Boston or Chicago or Glendora or Richmond or London or Auckland
lei16k-lei19k/mo HybridFull Time
5+ YOEBachelor's in CS/Data/Math, 5+ years software engineering with focus on testing/evaluation or AI/ML tooling, strong Python, experience building evaluation pipelines and LLM eval concepts, familiarity with RAG and production data pipelines.
RAGAS, LangSmith, PromptFlow, Snowflake, Python, RAG
1mo
Save
Mark Applied
Hide
AI Data Analytics Engineer
Fort Collins, Colorado, United States
$102k-$147k/yr OnsiteFull Time
BillGO: Digital bill payment network for businesses and consumers.
3+ YOE3+ years building production data or software products with analytics engineering and applied AI; proficiency in Python and SQL; experience with data warehouses, dbt, LLMs, and model evaluation.
Python, SQL, Snowflake, BigQuery, Redshift, dbt, LLM APIs, vector databases, embeddings, LangChain, LangGraph, Retrieval-Augmented Generation (RAG)
1w
Save
Mark Applied
Hide
AI/ML Engineer - Agentic AI (Remote)
United States or Englewood
$132k-$150k/yr RemoteFull Time
8+ YOERequires 8+ years in AI/ML, data science, or software engineering; expertise in LLMs, Agentic AI, RAG, Python, AI frameworks, evaluation, scalable architecture, and MLOps/LLMOps.
LLMs, RAG, OpenAI, Azure OpenAI, Anthropic, Gemini, Python, LangChain, LangGraph, Semantic Kernel, LlamaIndex, MLOps, LLMOps, Azure, AWS, Google Cloud
1w
Save
Mark Applied
Hide
Agentic AI Engineer
Denver, Colorado, United States
$95k-$135k/yr HybridFull Time
1.5+ YOEProduction agentic AI systems, strong Python, RAG pipelines, vector databases, LLM evaluation, agent frameworks, LLM fundamentals, and cloud platform experience; GCP preferred.
Python, GCP, AWS, Azure, Neo4j, LangChain, LangGraph, CrewAI, AutoGen, Google Agent Development Kit (ADK), Vertex AI Agent Builder, Model Context Protocol (MCP), JanusGraph, LangSmith, Langfuse, RAG
2w
Save
Mark Applied
Hide
Staff AI Platform Engineer
San Francisco or Los Angeles or Denver or Austin or Chicago or New York or Seattle or Toronto or Santa Barbara or San Diego
$180k-$273k/yr RemoteFull Time
7+ YOERequires 7+ years in AI/ML platform engineering, strong Python and/or TypeScript, distributed systems, APIs, evaluation, observability, and AI infrastructure experience; bachelor's degree or equivalent practical experience.
Python, TypeScript, JavaScript, React, Kubernetes, Docker, Terraform, LangChain, LangGraph, LlamaIndex, LiveKit, REST, gRPC, MCP, A2A
3w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1mo
Save
Mark Applied
Hide
Senior AI Platform Engineer
San Francisco or Alabama or Arkansas or California or Colorado or Florida or Georgia or Illinois or Kentucky or Massachusetts or Michigan or Montana or Missouri or Nevada or New Jersey or New York or North Carolina or Oregon or Pennsylvania or Texas or Washington or Utah or Virginia or Wisconsin
$156k-$231k/yr HybridFull Time
3+ YOE3+ years building production software or ML/data systems; strong Python and systems/API design; experience with LLM retrieval/RAG, agents, evaluation, Databricks/MLflow and modern data stacks.
Python, PySpark, dbt, Databricks, Delta Lake, Unity Catalog, MLflow, Astronomer, Airflow, Claude, GitHub, Shortcut, Notion, pgvector, Pinecone, Weaviate, LangGraph, MCP
3w
Save
Mark Applied
Hide
AI / ML Engineer
United States or Brownsville or Madison or Boise or Lincoln or Centennial
$95k-$130k/yr HybridFull Time
2+ YOERequires 2+ years in AI, ML, or software engineering; Python, AWS, agentic LLM systems, evaluation, production deployment, RAG, CI/CD, and Terraform experience; bachelor's or master's degree preferred; U.S. citizenship and ability to obtain 6C clearance.
Python, AWS, Amazon Bedrock, AWS Lambda, Amazon ECS, AWS Fargate, Amazon S3, Amazon OpenSearch, AWS Step Functions, Terraform, Docker, Kubernetes, CI/CD, RAG
4w
Save
Mark Applied
Hide
Sr AI Context Engineer
Lee's Summit or Missouri or Alaska or Hawaii or California or Washington or Oregon or Colorado or Wyoming or Montana or New York or Connecticut or Vermont or Pennsylvania or Maine or United States
$125k-$158k/yr RemoteFull Time
5+ YOE5+ years backend/software or applied ML experience, 1–2 years GenAI/RAG focus, strong Python and SQL, experience with vector search, document parsing, HIPAA-compliant handling of PHI, and retrieval evaluation.
Python, SQL, Airflow, Prefect, Temporal, LlamaIndex, Unstructured.io, LangChain, Pinecone, pgvector, Azure AI Search, Weaviate, Qdrant, Ragas, TruLens, Arize Phoenix
1w
Save
Mark Applied
Hide
AI Governance and Evaluation Manager (Remote From Anywhere In CO)
Colorado, United States
$145k-$160k/yr RemoteFull Time, Temporary
5+ YOERequires 5+ years working across multidisciplinary teams, including data science, engineering, policy, and operations, plus 1+ year working with generative AI models and demonstrated delivery of scalable solutions.
NIST AI Risk Management Framework, Agile
2mo
Save
Mark Applied
Hide
Principal Developer, AI Networking
Santa Clara or Texas or Colorado or Washington
$272k-$489k/yr OnsiteFull Time
15+ YOEBSc in CS or equivalent; 15+ years in high-performance networking (RDMA, MPI, NCCL, SHARP); experience with CUDA, PyTorch/TensorFlow; proficiency in Python, Bash, C++; performance evaluation and container environments.
RDMA, MPI, NCCL, SHARP, CUDA, TensorFlow, PyTorch, Python, Bash, C++
2mo
Save
Mark Applied
Hide
AI Tools Architect / Tech Lead
Spring or Fort Collins
$152k-$349k/yr HybridFull Time
10+ YOE10+ years software engineering; hands-on AI coding tools; technical leadership; CI/CD familiarity; strong communication; adaptable; AI model evaluation knowledge.
GitHub Copilot, ChatGPT, Claude, Python, CI/CD
2d
Save
Mark Applied
Hide
AI Business Partner
Denver, Colorado, United States
$110k-$200k/yr HybridFull Time
Requires hands-on low-code AI solution development, business analysis, GenAI and prompt engineering, stakeholder management, change adoption, documentation, and evaluation of emerging AI tools.
Microsoft Copilot Studio, Microsoft Power Automate, Microsoft Power Apps, Microsoft 365, Claude Desktop, SharePoint, Microsoft Teams, Dataverse, Azure AI services
2mo
Save
Mark Applied
Hide
Senior Applied AI Solutions Architect — Amazon Connect, Applied AI SA - AIVT
Seattle or Arlington or Atlanta or New York or Denver or Herndon or Chicago or Mountain View or San Francisco
$154k-$239k/yr OnsiteFull Time
7+ YOE7+ years in technology domains; 3+ years in design/implementation; familiarity with MCP/A2A; AI/ML concepts incl. LLMs, prompt engineering, RAG, model evaluation
Python, Node.js, AWS Lambda, API Gateway, Step Functions, Bedrock, MCP, RAG, DynamoDB, RDS, S3, OpenSearch, Kendra
1d
Save
Mark Applied
Hide
Senior AI Applied Manager
San Francisco or Chicago or Denver or Los Angeles or Phoenix
$125k-$186k/yr OnsiteFull Time
8+ YOERequires 8+ years of software engineering experience, expert Python, strong AWS and data skills, LLM evaluation and agentic systems experience, communication ability, and a bachelor's degree in computer science or engineering.
Python, AWS, SQL, Dataiku (DSS), OpenAI AgentKit, ChatKit, Apps SDK/MCP
2mo
Save
Mark Applied
Hide
Space-to-Ground Systems Integration Engineer
El Segundo or Chantilly or Colorado Springs
$100k-$176k/yr OnsiteFull Time
5+ YOEBachelor's in engineering/physics/computer science (or related); 5+ years Test & Evaluation and space systems integration experience (8+ for specialist); secret clearance required (active preferred or ability to obtain); experience with requirements flow-down, SIP/SEP/STIP, test planning/execution, cloud (AWS, Microsoft Azure, GCP), systems enginee...
Cameo, STK, Simulink, Python, MATLAB, AWS, Microsoft Azure, GCP, CI/CD, DevSecOps, AI/ML
2w
Save
Mark Applied
Hide
Senior Product Manager, AI & Data Science Products
Seattle or Richmond or Austin or North Charleston or Philadelphia or San Francisco or Denver or Miami or Atlanta or Chicago or Augusta or Boston or Las Vegas or Trenton or New York or Raleigh or Portland
RemoteFull Time
3+ YOERequires 3+ years of product management or comparable experience, customer-facing AI or data product ownership, applied AI/ML knowledge, data science and engineering collaboration, experimentation, analytics, and quality evaluation skills.
AI, machine learning, APIs, MCP
2d
Save
Mark Applied
Hide
Sales Engineer - Mid Market - West
Washington or Oregon or California or Nevada or Arizona or Idaho or Montana or Wyoming or Colorado or Utah or New Mexico or Alaska or Hawaii or Texas or Oklahoma or Kansas or Nebraska or South Dakota or North Dakota or Minnesota or Iowa or Missouri or Arkansas or Louisiana
$96k-$145k/yr RemoteFull Time
2+ YOEBachelor's degree required; 2+ years in customer-facing pre-sales, success, or support. Requires technical consulting, cloud, networking, integrations, hardware/software evaluation, and U.S. work authorization.
Internet of Things (IoT), AI, Bash, Python, JavaScript, Raspberry Pi, Arduino, Controller Area Networks (CAN bus), Power Take Off (PTO)
3w
Save
Mark Applied
Hide
Senior Manager, Applied Science
Illinois or Colorado or Virginia or Arizona or Alabama or Washington or Texas or California
$208k-$312k/yr HybridFull Time
6+ YOE1+ MgmtMaster's/PhD or equivalent, 6+ years applied AI/ML experience including production deployment, 1+ year people leadership, generative AI fluency, strong software engineering and evaluation skills.
Artificial Intelligence (AI), Machine Learning (ML), Business Intelligence (BI), Data Analysis, Database Management, Data Governance, Data Visualization, Information Management
2mo
Save
Mark Applied
Hide
Content Automation and GenAI - Consulting and Delivery Manager
New York City or Milwaukee or Herndon or Dallas or Columbus or Kirkland or Irvine or Cincinnati or Cleveland or Oklahoma City or Austin or Albany or Chicago or Arlington or St. Petersburg or Hartford or Pittsburgh or St. Louis or Miami or Sacramento or Raleigh or Minneapolis or Mountain View or Scottsdale or Morristown or San Francisco or Denver or Boston or Philadelphia or Des Moines or Overland Park or Los Angeles or Charlotte or Walnut Creek or Carmel or Seattle or Houston or Atlanta or Redmond or Bentonville or Beaverton or Nashville or Detroit or San Diego
$80k-$266k/yr OnsiteFull Time
7+ YOE7+ years client-facing consulting or digital transformation experience; hands-on experience evaluating/applying content automation or GenAI tools and delivering GenAI POCs; strong understanding of LLM concepts, prompt engineering, safety/responsible AI, content operations, and team leadership.
OpenAI, Google Gemini, M365 Copilot, Writer, FireFly, ChatGPT, Adobe Workfront, AEM, CMS, DAM, PIM
Explore Jobs
Expand Your Job Search
Top Cities in Colorado for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Denver, CO
- AI Evaluation Engineer jobs in Colorado Springs, CO
- AI Evaluation Engineer jobs in Aurora, CO
- AI Evaluation Engineer jobs in Fort Collins, CO
- AI Evaluation Engineer jobs in Lakewood, CO
- AI Evaluation Engineer jobs in Thornton, CO
- AI Evaluation Engineer jobs in Westminster, CO
- AI Evaluation Engineer jobs in Arvada, CO
- AI Evaluation Engineer jobs in Centennial, CO
- AI Evaluation Engineer jobs in Pueblo, CO
- AI Evaluation Engineer jobs in Boulder, CO
- AI Evaluation Engineer jobs in Greeley, CO
- AI Evaluation Engineer jobs in Longmont, CO
- AI Evaluation Engineer jobs in Loveland, CO
- AI Evaluation Engineer jobs in Broomfield, CO
- AI Evaluation Engineer jobs in Grand Junction, CO
- AI Evaluation Engineer jobs in Castle Rock, CO
- AI Evaluation Engineer jobs in Commerce City, CO
- AI Evaluation Engineer jobs in Parker, CO
- AI Evaluation Engineer jobs in Littleton, CO