28 ai evaluation engineer jobs at 21 companies in North Carolina
2w
Save
Mark Applied
Hide
Observability & Evaluation Engineer
Charlotte, North Carolina, United States
HybridFull Time
7+ YOERequires 7+ years of engineering experience, 5+ years of Python, observability, LLM evaluation, Agile, and production support experience; knowledge of RAG, AI quality, telemetry, and monitoring.
Python, Kubernetes, CI/CD
1mo
Save
Mark Applied
Hide
Evaluation and ML Systems Engineer, AI Safety and Security Engineering
Santa Clara or North Carolina or Tennessee or New York or California or Florida
$152k-$242k/yr RemoteFull Time
5+ YOEBachelor's degree or equivalent experience and 5+ years in ML engineering or evaluation. Requires benchmark design, statistically sound comparisons, Python engineering, experiment tracking, and data pipelines.
Python
1w
Save
Mark Applied
Hide
Applied AI Engineer
Canada or San Francisco or Raleigh or Boston
HybridFull Time
Applied GenAI experience with prompt engineering, structured outputs, RAG, evaluation datasets, production LLM features, multimodal inputs, and model quality, cost, and latency optimization.
LLMX, Attachments, OCR, RAG
3mo
Save
Mark Applied
Hide
Lead AI Engineer — Productivity Systems
Durham or Miami or Palo Alto or Washington
$12k-$18k/mo HybridFull Time
Proven experience shipping LLM-powered systems in production; strong prompt/context engineering, evaluation and guardrail development, and software engineering skills (Python/TypeScript/Clojure).
Python, TypeScript, Clojure, n8n, Zapier, AWS, Claude Code, Cursor, Copilot, Slack, Google Workspace, Jira, Terraform, RAG, MCP
3w
Save
Mark Applied
Hide
AI Engineer - NC
North Carolina, United States
OnsiteFull Time
5+ YOE5+ years of AI/ML engineering experience evaluating, fine-tuning, and deploying LLMs in production; cloud infrastructure, RAG, embeddings, vector databases, agentic workflows, EvalOps, and production software engineering required.
LLM, AWS, GCP, RAG, EvalOps, Evals-as-a-Service, vector databases
2mo
Save
Mark Applied
Hide
AI Agent Engineer
Charlotte or San Francisco or Los Angeles or New York City
$95k-$120k/yr OnsiteFull Time
4+ YOEBachelor's degree or equivalent experience, 4+ years in MLOps/AI/ML engineering or data engineering, hands-on Databricks and MLflow experience, strong Python and SQL skills, experience with agent frameworks and evaluation/monitoring of non-deterministic AI systems.
Databricks, Mosaic AI, Databricks Agent Framework, Databricks Vector Search, MLflow, Python, LangChain, LlamaIndex, AutoGen, SQL, Delta Lake, Unity Catalog, CI/CD, Sigma, Microsoft Power BI, Streamlit, Gradio
5d
Save
Mark Applied
Hide
AI Engineer
New York City or United States or Raleigh-Durham or Atlanta or Philadelphia
$150k-$300k/yr HybridFull Time
Production experience launching LLM or agent products, strong Python, agent frameworks, RAG, prompt engineering, evaluation systems, ML tools, orchestration workflows, and stakeholder communication.
Python, AWS, PyTorch, scikit-learn, SQL, Spark, Airflow, ZenML, Kedro, dbt
1mo
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1mo
Save
Mark Applied
Hide
Senior AI Platform Engineer
San Francisco or Alabama or Arkansas or California or Colorado or Florida or Georgia or Illinois or Kentucky or Massachusetts or Michigan or Montana or Missouri or Nevada or New Jersey or New York or North Carolina or Oregon or Pennsylvania or Texas or Washington or Utah or Virginia or Wisconsin
$156k-$231k/yr HybridFull Time
3+ YOE3+ years building production software or ML/data systems; strong Python and systems/API design; experience with LLM retrieval/RAG, agents, evaluation, Databricks/MLflow and modern data stacks.
Python, PySpark, dbt, Databricks, Delta Lake, Unity Catalog, MLflow, Astronomer, Airflow, Claude, GitHub, Shortcut, Notion, pgvector, Pinecone, Weaviate, LangGraph, MCP
2mo
Save
Mark Applied
Hide
AI Operations Engineer (Durham, NC, US)
Durham, North Carolina, United States
HybridFull Time
Experience with AI platforms, evaluating AI outputs, dataset curation, prompts, and integrating LLM-powered systems; strong analytical skills; hybrid work in Durham.
AI platforms, LLM systems, vector search, RAG pipelines
1d
Save
Mark Applied
Hide
Senior AI Platform Engineer
Charlotte, North Carolina, United States
HybridFull Time
7+ YOE7+ years building and operating production software or automation systems, including 3+ years resolving production failures; Python or equivalent coding, API and AI/LLM integration, cloud identity, evaluation, and production ownership experience.
Python, Microsoft Copilot, Microsoft Copilot Studio, Microsoft Graph API, Microsoft SharePoint, Microsoft Teams, Microsoft Power Platform, Model Context Protocols (MCPs), Microsoft Azure, Amazon Web Services, Azure AI Foundry, REST APIs, APIs, webhooks
1mo
Save
Mark Applied
Hide
Senior Forward Deployed AI Engineer (GenAI, AWS)
New Jersey or New York or Pennsylvania or North Carolina or Connecticut or Massachusetts or Virginia
RemoteFull Time
8+ YOE8+ years building production software, hands-on coding (Python/TypeScript), cloud-native AWS delivery, shipped GenAI/LLM systems to production, built evaluation suites, strong stakeholder communication and domain learning ability.
Claude, Claude Code, Cowork, AWS, Python, TypeScript, Kubernetes, ECS, PyTorch, SageMaker, MLflow, Neo4j, AWS Neptune, AWS CDK, CloudFormation, Terraform, ChatGPT
1w
Save
Mark Applied
Hide
Sr ML/AI Engineer
Arizona or Georgia or Idaho or Illinois or Florida or Massachusetts or Maine or Michigan or Missouri or North Carolina or New York or Pennsylvania or Tennessee or Texas or Oregon or Virginia or Washington or Durham
RemoteFull Time
Bachelor’s degree in a quantitative field, significant hands-on ML experience, advanced machine learning and applied statistics expertise, strong Python skills, and experience productionizing, monitoring, and evaluating models.
Python, Snowflake, Snowpark ML, Snowflake Feature Store, Snowflake Model Registry, AWS, LLM, retrieval-augmented generation
1mo
Save
Mark Applied
Hide
Industrial Engineer
Whitsett, North Carolina, United States
$75k-$115k/yr OnsiteFull Time
3+ YOEBachelor’s degree in engineering or related field, 3–5 years of manufacturing experience, lean manufacturing and process expertise, ERP proficiency preferably Oracle, and ability to use and evaluate AI tools.
Oracle, ERP, AI, 5S, TIMWOOD, poka-yoke, OEE
3w
Save
Mark Applied
Hide
Manager, Conversational AI
United States or California or Florida or Georgia or Idaho or Illinois or Massachusetts or Michigan or New Jersey or New York or North Carolina or Ohio or Oregon or Pennsylvania or Texas or Virginia or Washington or Wisconsin
RemoteFull Time
Experience leading teams and building conversational AI products, including LLM-powered agents, evaluation frameworks, optimization, experimentation, and cross-functional delivery with Product, Engineering, Data, and AI teams.
AI, Machine Learning, LLM
3w
Save
Mark Applied
Hide
Technical Support Engineer - AI Lead
Redmond or Fargo or Las Colinas or Charlotte or San Francisco or New York City
$131k-$277k/yr HybridFull Time
10+ YOEBachelor's degree in computer science, IT, or related field and 10+ years of technical support, consulting, or IT experience; expertise in AI/ML, governance, evaluation, and executive stakeholder management.
AI, Responsible AI, LLM, agentic, AI/ML
2mo
Save
Mark Applied
Hide
VP Mgr-Applied AI Engineering
Charlotte, North Carolina, United States
$169k-$245k/yr OnsiteFull Time
Deep LLM and AI architecture expertise; leadership of technical teams; AI evaluation, routing, fine-tuning, DevOps, infrastructure, governance, and responsible AI experience; bachelor's or master's degree in a technical field.
LLMs, UI/UX, DevOps, CI/CD, infrastructure as code
2mo
Save
Mark Applied
Hide
Agentic AI Architect - Intelligence Engineering (East)
Atlanta or Boston or Charlotte or Hartford or Miami or New Brunswick or New York City or Philadelphia or Raleigh or Washington
$108k-$218k/yr HybridFull Time
5+ YOE5+ years software engineering experience with 2+ years on generative/agentic AI; experience designing multi-agent systems, RAG, cloud AI platforms, Python and API frameworks, evaluation and governance for production AI systems.
Amazon Web Services, Azure, Google Cloud, AWS Bedrock, Azure AI Foundry, OpenAI Service, GCP Vertex AI, Gemini, Databricks, Lang Graph, Strands Agents SDK, OpenAI Agents SDK, Google ADK, Model Context Protocol (MCP), LangSmith, RAGAS, DeepEval, Claude Code, Cursor, Kiro, Python, FastAPI, Flask
1w
Save
Mark Applied
Hide
Director, GTM Sales - AI Operations
North Carolina, United States
$201k-$258k/yr RemoteFull Time
8+ YOE8+ years in GTM technology, sales engineering, or revenue systems; seller perspective; LLM or agentic AI development and evaluation; small-team leadership; Salesforce, Gong, enablement stack; bachelor's degree or equivalent.
Salesforce, Gong, AI Assistant, AI Tutor, MindTickle, Seismic, LangGraph, Python
1d
Save
Mark Applied
Hide
Senior AI Systems Architect
New York City or Chicago or Boston or San Francisco or United States or Toronto or Arizona or California or Colorado or Florida or Georgia or Illinois or Massachusetts or North Carolina or New Jersey or Nevada or New York or Oregon or Tennessee or Texas or Utah or Virginia or Ontario
$160k-$210k/yr RemoteFull Time
5+ YOE5+ years in product or forward-deployed engineering, preferably at an early-stage startup, with hands-on experience deploying LLM systems and managing evaluations, latency, cost, regressions, and failures.
OpenAI, ChatGPT, LLM
Explore Jobs
Expand Your Job Search
Top Cities in North Carolina for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Charlotte, NC
- AI Evaluation Engineer jobs in Raleigh, NC
- AI Evaluation Engineer jobs in Greensboro, NC
- AI Evaluation Engineer jobs in Durham, NC
- AI Evaluation Engineer jobs in Winston-Salem, NC
- AI Evaluation Engineer jobs in Fayetteville, NC
- AI Evaluation Engineer jobs in Cary, NC
- AI Evaluation Engineer jobs in Wilmington, NC
- AI Evaluation Engineer jobs in High Point, NC
- AI Evaluation Engineer jobs in Greenville, NC
- AI Evaluation Engineer jobs in Asheville, NC
- AI Evaluation Engineer jobs in Concord, NC
- AI Evaluation Engineer jobs in Gastonia, NC
- AI Evaluation Engineer jobs in Jacksonville, NC
- AI Evaluation Engineer jobs in Chapel Hill, NC
- AI Evaluation Engineer jobs in Rocky Mount, NC
- AI Evaluation Engineer jobs in Huntersville, NC
- AI Evaluation Engineer jobs in Burlington, NC
- AI Evaluation Engineer jobs in Wilson, NC
- AI Evaluation Engineer jobs in Kannapolis, NC