27 ai evaluation engineer jobs at 19 companies in Florida
3d
Save
Mark Applied
Hide
AI Evaluation Engineer (QA)
New York City or Austin or Miami or Dallas or Phoenix or São Paulo or Toronto
OnsiteFull Time, Contract
4+ YOEBachelor's degree in a technical field or equivalent experience, 4+ years in QA/test engineering, Python and data-science skills, LLM evaluation and statistical analysis experience, and automated test framework expertise.
Python, LLM evaluation frameworks, CI/CD, Test automation frameworks
2mo
Save
Mark Applied
Hide
Principal Test & Evaluation Engineer
Melbourne, Florida, United States
$89k-$134k/yr OnsiteFull Time
1+ YOESTEM degree (BS+5yrs | MS+3yrs | PhD+1yr), active US Secret clearance (with ability to obtain SAP), experience with virtualization/containerization, on-premise and cloud (Azure/AWS), Python scripting on Linux/Windows, Agile, demo/experiment integration, ability to lift 75 lbs, willingness to travel up to 25%.
Python, Linux, Windows, Azure, AWS, Kubernetes, O-RAN, 5G, AI/ML, Model-Based Systems Engineering (MBSE), Agile, microservices
2mo
Save
Mark Applied
Hide
Principal Test & Evaluation Engineer
Melbourne or United States
$89k-$134k/yr OnsiteFull Time
1+ YOESTEM degree (BS/MS/PhD) with 5/3/1 years respectively, active U.S. Secret clearance, experience with 5G/O-RAN, cloud (Azure/AWS), virtualization/containerization, Python on Linux/Windows, Agile, ability to lift 75 lbs, willingness to travel up to 25%.
Python, Linux, Windows, Azure, AWS, Kubernetes, O-RAN, Model-Based Systems Engineering (MBSE), AI/ML, Agile
1w
Save
Mark Applied
Hide
Applied AI Lead - Evaluation & Measurement
Frederick or Beavercreek or Fort Walton Beach or Washington
OnsiteFull Time
5+ YOERequires 5+ years in data, ML, or test engineering, including 3+ years in evaluation or model validation and recent hands-on LLM evaluation. Requires a bachelor's degree or equivalent, U.S. citizenship, and active or obtainable DOD clearance.
pipelines, dashboards, large-language-model, telemetry
1mo
Save
Mark Applied
Hide
Staff AI Engineer
Boca Raton, Florida, United States
HybridFull Time
10+ YOEMaster's or PhD in CS/Software Engineering,10+ years in ML/AI or software engineering with production agentic AI experience,4+ years senior/staff-level leadership,expertise in distributed AI systems and evaluation.
Model Context Protocol (MCP), RAG, LLM, vector database, GPU, CI
3w
Save
Mark Applied
Hide
AI Engineer - FL
Florida, United States
OnsiteFull Time
5+ YOE5+ years of AI/ML engineering experience evaluating, fine-tuning, and deploying production LLMs; AWS or GCP experience; RAG, embedding models, vector databases, agentic workflows, and evaluation frameworks.
LLM, Evals, EvalOps, AWS, GCP, RAG, vector databases
3mo
Save
Mark Applied
Hide
Lead AI Engineer — Productivity Systems
Durham or Miami or Palo Alto or Washington
$12k-$18k/mo HybridFull Time
Proven experience shipping LLM-powered systems in production; strong prompt/context engineering, evaluation and guardrail development, and software engineering skills (Python/TypeScript/Clojure).
Python, TypeScript, Clojure, n8n, Zapier, AWS, Claude Code, Cursor, Copilot, Slack, Google Workspace, Jira, Terraform, RAG, MCP
4w
Save
Mark Applied
Hide
Applied AI Engineer, Founding Team
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$63k-$219k/yr OnsiteFull Time
5+ YOERequires 5+ years in software engineering, AI/ML, or enterprise technology; experience with retrieval systems, agentic AI, MCP, evaluation frameworks, and a bachelor's degree or equivalent.
Oracle 23ai, Model Context Protocol (MCP), Claude, Anthropic, Oracle EPM, Oracle ERP, Oracle Cloud Infrastructure (OCI)
1mo
Save
Mark Applied
Hide
Senior AI Platform Engineer
San Francisco or Alabama or Arkansas or California or Colorado or Florida or Georgia or Illinois or Kentucky or Massachusetts or Michigan or Montana or Missouri or Nevada or New Jersey or New York or North Carolina or Oregon or Pennsylvania or Texas or Washington or Utah or Virginia or Wisconsin
$156k-$231k/yr HybridFull Time
3+ YOE3+ years building production software or ML/data systems; strong Python and systems/API design; experience with LLM retrieval/RAG, agents, evaluation, Databricks/MLflow and modern data stacks.
Python, PySpark, dbt, Databricks, Delta Lake, Unity Catalog, MLflow, Astronomer, Airflow, Claude, GitHub, Shortcut, Notion, pgvector, Pinecone, Weaviate, LangGraph, MCP
2w
Save
Mark Applied
Hide
Senior AI Engineer
Sarasota, Florida, United States
RemoteFull Time
5+ YOERequires 5+ years of backend engineering experience in a statically typed language, production LLM features, LLM serving, retrieval systems, API design, evaluation, and US work authorization.
Go, Java, Kotlin, C#, Python, LiteLLM, vLLM, TGI, Qdrant, pgvector, LoRA, Ruby on Rails, React, TypeScript, PostgreSQL, Redis, RabbitMQ
2mo
Save
Mark Applied
Hide
Senior AI Engineer
United States or New York City or Palm Beach or Los Angeles or Miami or Greenwich or Westchester County or The Hamptons or San Francisco or Manhattan
$190k-$250k/yr RemoteFull Time
4+ YOE4+ years building and operating LLM-powered production ML systems, strong Python, backend framework experience (FastAPI/Flask), LLM orchestration and retrieval design, Snowflake/dbt/Dagster/Airflow familiarity, evaluation systems, HIPAA/PHI knowledge, and strong written communication.
Python, FastAPI, Flask, Snowflake, dbt, Dagster, Airflow, vLLM, Triton, TGI, SGLang, LOINC, SNOMED CT, ICD-10, RxNorm, FHIR, HL7
3d
Save
Mark Applied
Hide
Senior Agentic (AI) Engineer
Tampa or Orlando
RemoteFull Time
5+ YOERequires 5+ years of software engineering experience, including 2+ years building production LLM or agentic systems, strong Python, TypeScript/Node.js, agent frameworks, RAG, evaluations, MLOps, AWS, and regulated-finance experience.
Python, Node.js, TypeScript, LangGraph, CrewAI, AutoGen, Claude Agent SDK, MCP, OpenAI SDK, Anthropic Claude, OpenAI, PostgreSQL, pgvector, OpenSearch, Kafka, Redshift, Redis, AWS, Kubernetes, EKS, ArgoCD, Terraform, LangSmith, Langfuse, Braintrust, DataDog, MSK, RDS, S3, Lambda, APIs, SOC 2, GDPR, CCPA, P90/P99
4w
Save
Mark Applied
Hide
Application Software Engineer, Applied AI
Bastrop or Cape Canaveral or Hawthorne or McGregor or Redmond or Brownsville or Vandenberg
OnsiteFull Time
1+ YOEDesign, evaluate, deploy, and integrate AI systems including LLMs; 1+ years full-stack experience; bachelor's in CS/engineering/math or 2+ years professional software experience; strong software engineering and AI production experience.
Python, C#.NET, Go, Scala, Java, Docker, Kubernetes, Kafka, Angular, React, PostgreSQL, SQL Server, LLM
3w
Save
Mark Applied
Hide
Manager, Conversational AI
United States or California or Florida or Georgia or Idaho or Illinois or Massachusetts or Michigan or New Jersey or New York or North Carolina or Ohio or Oregon or Pennsylvania or Texas or Virginia or Washington or Wisconsin
RemoteFull Time
Experience leading teams and building conversational AI products, including LLM-powered agents, evaluation frameworks, optimization, experimentation, and cross-functional delivery with Product, Engineering, Data, and AI teams.
AI, Machine Learning, LLM
1w
Save
Mark Applied
Hide
Senior Electrical Engineer – MV AI UPS Data Centers & Storage
Alpharetta or Atlanta or Miami or Tysons or United States
RemoteFull Time
8+ YOERequires 8+ years of electrical engineering experience focused on battery systems, BESS evaluation, inverter/PCS validation, EMS or Plant Controller integration, SCADA, PSCAD, MATLAB, and engineering analysis tools.
Python, VS Code, Jupyter Notebooks, Git, PSCAD, MATLAB, SCADA, EMS
1mo
Save
Mark Applied
Hide
Agentic AI Architect - Intelligence Engineering (East)
Atlanta or Boston or Charlotte or Hartford or Miami or New Brunswick or New York City or Philadelphia or Raleigh or Washington
$108k-$218k/yr HybridFull Time
5+ YOE5+ years software engineering experience with 2+ years on generative/agentic AI; experience designing multi-agent systems, RAG, cloud AI platforms, Python and API frameworks, evaluation and governance for production AI systems.
Amazon Web Services, Azure, Google Cloud, AWS Bedrock, Azure AI Foundry, OpenAI Service, GCP Vertex AI, Gemini, Databricks, Lang Graph, Strands Agents SDK, OpenAI Agents SDK, Google ADK, Model Context Protocol (MCP), LangSmith, RAGAS, DeepEval, Claude Code, Cursor, Kiro, Python, FastAPI, Flask
8h
Save
Mark Applied
Hide
Senior AI Systems Architect
New York City or Chicago or Boston or San Francisco or United States or Toronto or Arizona or California or Colorado or Florida or Georgia or Illinois or Massachusetts or North Carolina or New Jersey or Nevada or New York or Oregon or Tennessee or Texas or Utah or Virginia or Ontario
$160k-$210k/yr RemoteFull Time
5+ YOE5+ years in product or forward-deployed engineering, preferably at an early-stage startup, with hands-on experience deploying LLM systems and managing evaluations, latency, cost, regressions, and failures.
OpenAI, ChatGPT, LLM
2w
Save
Mark Applied
Hide
Gen AI Engineering Analyst - Vice President
Jacksonville, Florida, United States
$114k-$171k/yr HybridFull Time
6+ YOE6–10 years in application development or systems analysis; Python, Java/C++, SQL, AI/ML frameworks, APIs, containers, cloud platforms, NLP, data libraries, visualization, Git, and model evaluation skills.
Python, Java, C++, SQL, TensorFlow, PyTorch, Scikit-learn, REST, GraphQL, Django, Flask, Docker, Kubernetes, Pandas, NumPy, Apache Spark, AWS, Azure, Google Cloud AI, AWS SageMaker, Azure ML, NLTK, spaCy, Hugging Face Transformers, Power BI, Tableau, Matplotlib, Git, GitHub
2mo
Save
Mark Applied
Hide
Gen AI Engineering Analyst - Vice President
Jacksonville, Florida, United States
$114k-$171k/yr HybridFull Time
6+ YOE6-10 years in apps development or systems analysis; strong Python, Java/C++, SQL; ML frameworks; API/web services; Docker/Kubernetes; data libraries; cloud AI/ML; NLP; data visualization; Git; model evaluation metrics
Python, Java, C++, SQL, TensorFlow, PyTorch, Scikit-learn, Docker, Kubernetes, Pandas, NumPy, Apache Spark, AWS, Azure, Google Cloud, SageMaker, NLTK, spaCy, Hugging Face Transformers, Power BI, Tableau, Matplotlib, Git, GitHub
2w
Save
Mark Applied
Hide
Senior Product Manager, AI & Data Science Products
Seattle or Richmond or Austin or North Charleston or Philadelphia or San Francisco or Denver or Miami or Atlanta or Chicago or Augusta or Boston or Las Vegas or Trenton or New York or Raleigh or Portland
RemoteFull Time
3+ YOERequires 3+ years of product management or comparable experience, customer-facing AI or data product ownership, applied AI/ML knowledge, data science and engineering collaboration, experimentation, analytics, and quality evaluation skills.
AI, machine learning, APIs, MCP
Explore Jobs
Expand Your Job Search
Top Cities in Florida for AI Evaluation Engineer
- AI Evaluation Engineer jobs in Jacksonville, FL
- AI Evaluation Engineer jobs in Miami, FL
- AI Evaluation Engineer jobs in Tampa, FL
- AI Evaluation Engineer jobs in Orlando, FL
- AI Evaluation Engineer jobs in St. Petersburg, FL
- AI Evaluation Engineer jobs in Hialeah, FL
- AI Evaluation Engineer jobs in Tallahassee, FL
- AI Evaluation Engineer jobs in Fort Lauderdale, FL
- AI Evaluation Engineer jobs in Cape Coral, FL
- AI Evaluation Engineer jobs in Pembroke Pines, FL
- AI Evaluation Engineer jobs in Port Saint Lucie, FL
- AI Evaluation Engineer jobs in Hollywood, FL
- AI Evaluation Engineer jobs in Gainesville, FL
- AI Evaluation Engineer jobs in Miramar, FL
- AI Evaluation Engineer jobs in Coral Springs, FL
- AI Evaluation Engineer jobs in Palm Bay, FL
- AI Evaluation Engineer jobs in Miami Gardens, FL
- AI Evaluation Engineer jobs in Clearwater, FL
- AI Evaluation Engineer jobs in Pompano Beach, FL
- AI Evaluation Engineer jobs in West Palm Beach, FL