HiringCafe
Join our community

852 spark engineer jobs at 297 companies in California

1mo
Save
Mark Applied
Hide
Software Engineer, Spark Platform
San Francisco or Seattle or Sunnyvale or New York City
$131k-$285k/yr HybridFull Time
DoorDash
DoorDashNYSE: DASH: Technology enabling local on-demand delivery and commerce.
24+ YOERequires a CS degree or equivalent, 24+ years operating production distributed systems, Spark at scale, Kubernetes, cloud infrastructure, observability, and professional Python, Go, Scala, or Java with SQL fluency.
Apache Spark, Kubernetes, YuniKorn, Volcano, Kueue, Prometheus, OpenTelemetry, AWS, Python, Go, Scala, Java, SQL, Databricks, Amazon EMR
Job PostingView all
2mo
Save
Mark Applied
Hide
Software Engineer, Spark Platform
San Francisco or Sunnyvale or Seattle or New York City
$131k-$285k/yr HybridFull Time
DoorDash
DoorDashNYSE: DASH: Technology enabling local on-demand delivery and commerce.
24+ YOEExperience operating production distributed systems and Apache Spark at scale, Kubernetes production experience, familiarity with schedulers and observability stacks, cloud (AWS) experience, programming in Python/Go/Scala/Java, and SQL fluency. BS/MS/PhD in CS or equivalent.
Apache Spark, Amazon EMR, Databricks, Kubernetes, YuniKorn, Volcano, Kueue, Spark-on-Kubernetes operator, Prometheus, OpenTelemetry, Python, Go, Scala, Java, SQL, VPC
Job PostingView all
8h
Save
Mark Applied
Hide
Senior Staff Data Platform Engineer - Kafka - Apache Iceberg - Apache Spark
San Diego or San Francisco or Pleasanton or Santa Clara
$181k-$317k/yr HybridFull Time
ServiceNow
ServiceNowNYSE: NOW: Enterprise software providing cloud-based workflow automation platforms.
6+ YOERequires 10+ years of software development with a bachelor's, 8+ with a master's, 6+ with a PhD, or equivalent; expertise in Java, distributed systems, Kafka, Data Lakes, Apache Iceberg, Apache Spark, and ingestion pipelines.
Java, JVM, Kafka, Oracle, MySQL, PostgreSQL, Apache Iceberg, Apache Spark, Parquet, ORC, Avro
Job PostingView all
4w
Save
Mark Applied
Hide
Software Engineer - Infrastructure (Senior+)
San Francisco or New York
$100k-$300k/yr OnsiteFull Time
Cogent
Cogent: AI cybersecurity providing autonomous agents for enterprise vulnerability management.
5+ YOE5+ years infrastructure engineering experience, hands-on AWS, Clickhouse, Postgres, Spark; startup experience; security and on-prem deployment experience; able to drive large cross-stakeholder projects.
AWS, Clickhouse, Clickhouse Cloud, Postgres, Spark, GCP, Azure, Envoy, Istio, Hashicorp Vault, MicroVM (Runloop), SPIFFE, SPIRE
Job PostingView all
2mo
Save
Mark Applied
Hide
Staff Fullstack Engineer, Agentic Applications
Mountain View, California, United States
$192k-$260k/yr OnsiteFull Time
Databricks
Databricks: Data and AI software providing a unified platform.
8+ YOE8+ years software engineering with 2+ years building production LLM/agentic applications; deep Python fluency; experience with agentic frameworks, REST/GraphQL integrations, Databricks/Spark, and technical leadership.
Python, LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel, RAG pipelines, REST, GraphQL, Databricks Unity Catalog, Databricks, Spark, Workday, Greenhouse, ADP, MLflow
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer | Emerging Products
California or Colorado or Connecticut or Florida or Georgia or Illinois or Kansas or Massachusetts or Maryland or Maine or New Jersey or North Carolina or New York or Oregon or Tennessee or Texas or Virginia or Washington
$110k-$200k/yr HybridFull Time
NinjaOne
NinjaOne: Privately held NinjaOne provides endpoint management, patching, backup, and remote access software for IT teams and MSPs.
10+ YOE10+ years data engineering experience building lakehouse architectures and distributed systems; Bachelor's in CS or equivalent preferred; strong Python and SQL; expertise with Kafka, Spark, Airflow, Databricks, and open table formats.
Kafka, Spark, Airflow, Iceberg, Databricks, Delta, Hudi, Trino, Starburst, Python, SQL
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Platform Engineer
Austin or San Francisco or New York City or Seattle or Los Angeles or Chicago or Gurugram or Bengaluru
$170k-$230k/yr RemoteFull Time
SentiLink
SentiLink: Your fraud and risk partner
5+ YOE5+ years engineering experience; strong Python or Golang skills; building ETL/ELT pipelines at scale with Spark/Hadoop/Kafka; cloud (AWS/Azure/GCP) and database expertise; containerization and IaC experience.
Python, Golang, PostgreSQL (RDS), OpenSearch, Redshift, EMR, Spark, Docker, Lambda, AWS, EKS, SQS, SNS, S3, Kubernetes, Hadoop, Kafka, Microsoft Azure, GCP
Job PostingView all
2mo
Save
Mark Applied
Hide
Principal Engineer, Model Development Platform
Sunnyvale, California, United States
$296k-$335k/yr HybridFull Time
Wayve
Wayve: British autonomous-driving software licensing vehicle-agnostic AI Driver technology to automakers and fleet owners.
10+ YOE10+ years building large-scale distributed systems or ML infrastructure, 3+ years at staff/principal level, experience with Spark, Ray, Kubernetes, Airflow, MLflow, web frameworks, reliability engineering, and mentoring engineers.
Spark, Ray, Kubernetes, Airflow, MLflow, React, Flask, FastAPI
Job PostingView all
8h
Save
Mark Applied
Hide
Senior Staff Data Platform Engineer - Kafka - Apache Iceberg - Apache Spark
San Diego or San Francisco or Pleasanton or Santa Clara
$181k-$317k/yr HybridFull Time
ServiceNow
ServiceNowNYSE: NOW: Enterprise software providing cloud-based workflow automation platforms.
10+ YOERequires 10+ years with a bachelor's, 8+ with a master's, 6+ with a PhD, or equivalent; expert Java, distributed systems, Kafka, data lakes, Apache Iceberg, Apache Spark, databases, and data ingestion pipelines.
Java, JVM, Kafka, Oracle, MySQL, PostgreSQL, Apache Iceberg, Apache Spark, Parquet, ORC, Avro
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Los Angeles or San Francisco
$170k-$200k/yr HybridFull Time
Tatari
Tatari: Private ad-tech helping brands and agencies buy, measure, and optimize ads across linear and streaming TV.
5+ YOE5+ years building and operating production ETL pipelines, strong Python and SQL skills, Spark/PySpark, Databricks/Delta Lake, Airflow, data modeling and data quality expertise.
Python, SQL, Spark, PySpark, Databricks, Delta Lake, Airflow, ClickHouse
Job PostingView all
1mo
Save
Mark Applied
Hide
Staff Data Engineer
Denver or San Francisco
$190k-$264k/yr HybridFull Time
Checkr
Checkr: AI verification platform providing background, identity, employment, mortgage, tenant, and trust checks for businesses and individuals.
10+ YOE10+ years building scalable data platforms; expert PySpark, Python, SQL; experience with Kafka, Spark, Iceberg, data lakes, AWS; strong data modeling and security awareness.
PySpark, Python, SQL, MongoDB, Kafka, Spark, Iceberg, EKS, EMR, Serverless, Glue, Athena, S3, Databricks, Snowflake
Job PostingView all
1mo
Save
Mark Applied
Hide
Site Reliability Engineer, Compute Platform
San Jose, California, United States
$156k-$388k/yr OnsiteFull Time
TikTok
TikTok: Short-form mobile video and social media platform.
Bachelor's in CS/Engineering, strong Linux, networking, databases, Kubernetes, SRE/DevOps toolset knowledge, experience with ClickHouse/Spark/Presto/Doris/Hadoop, coding in Python/Shell/Java/Go, strong problem-solving and communication.
ClickHouse, Spark, Presto, Doris, Hadoop, Kubernetes, Python, Shell, Java, Go
Job PostingView all
4d
Save
Mark Applied
Hide
Senior Software Engineer — Distributed Compute / Spark Systems
Mountain View or San Francisco
$160k-$240k/yr OnsiteFull Time
Granica
Granica: AI research and products building data infrastructure and models for enterprises.
5+ YOEDeep distributed systems experience; production experience with distributed compute or query systems; performance optimization expertise; systems programming skills in Scala, Java, Go, Rust, or C++.
Apache Spark, Spark SQL, Trino, Presto, Flink, Databricks, Snowflake, Apache Iceberg, Delta Lake, Apache Hudi, Parquet, ORC, Amazon S3, GCS, ADLS, Scala, Java, Go, Rust, C++, EMR, Glue, Hive, Catalyst, Adaptive Query Execution, Velox, DuckDB, DataFusion
Job PostingView all
1mo
Save
Mark Applied
Hide
Hardware Analytics Engineer
Sunnyvale, California, United States
$214k-$225k/yr HybridFull Time
Cerebras Systems
Cerebras SystemsNasdaq Global Select Market: CBRS: Designs processors and systems for AI training and inference.
3+ YOEMaster's degree in EE/CE/CS or related +3 years experience; expertise in large-scale data pipelines, Hive/Spark, Python/SQL, Tableau, Linux, ML for hardware reliability and performance; strong statistical and A/B testing skills.
Python, SQL, Tableau, Hive, Spark, Linux
Job PostingView all
1mo
Save
Mark Applied
Hide
Site Reliability Engineer, Compute Platform
San Jose, California, United States
OnsiteFull Time
ByteDance
ByteDance: Global technology specializing in AI-powered content platforms.
Experience with Linux, networking, databases, Kubernetes, ClickHouse/Hadoop/Doris/Spark/Presto, scripting or programming (Python, Shell, Java, Go), incident management, and capacity planning.
ClickHouse, Spark, Presto, Doris, Hadoop, Kubernetes, Linux, Python, Shell, Java, Go
Job PostingView all
2w
Save
Mark Applied
Hide
Practice Customer Engineer, Data Analytics, Google Cloud
Sunnyvale or Atlanta or Austin or Boulder or Chicago or Kirkland or Los Angeles or New York City or Seattle or San Francisco or Portland or Phoenix or Playa Vista
$192k-$267k/yr OnsiteFull Time
Google
GoogleNASDAQ: GOOG, GOOGL: Global technology specializing in internet-related services and products.
10+ YOEBachelor's degree or equivalent, 10 years in cloud-native architecture, Big Data expertise, technical stakeholder engagement, and programming experience with Python, Spark, or SQL.
Google Cloud, Python, Spark, SQL, Informatica, Talend, Alooma, SAP, Data Services, Java, JavaScript, C++, Scala, R, Go, DNS, TCP, Linux, Microsoft Windows, VPN, VPC
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Machine Learning Engineer
San Jose, California, United States
HybridFull Time
Roku
RokuNASDAQ: ROKU: TV streaming platform powering the global television ecosystem.
5+ YOE5+ years applied machine learning experience; strong statistical, optimization, and experimental methodology knowledge; proficient in Spark, Python or Java; experience with real-time, low-latency systems and distributed ML frameworks.
Spark, Python, Java, Spark-MLlib, TensorFlow, Hive, Aerospike, ScyllaDB
Job PostingView all
2mo
Save
Mark Applied
Hide
Staff Machine Learning Engineer
Bengaluru or Noida or Santa Clara
HybridFull Time
Eightfold AI
Eightfold AI: AI talent intelligence platform provider helping employers recruit, retain, develop, and deploy their workforce.
6+ YOEDeep expertise in ML/DL/NLP and LLMs; technical leadership and team mentoring; proficiency with Python, TensorFlow, PyTorch, big data (Hadoop, Spark); experience deploying ML at scale. MS/PhD and 6+ years preferred.
Azure, AWS, Python, TensorFlow, PyTorch, Hadoop, Spark, C/C++, Hava, Scala, R, LLMs, Transformers
Job PostingView all
2mo
Save
Mark Applied
Hide
Principal Engineer, Model Development Platform
Sunnyvale, California, United States
$296k-$335k/yr HybridFull Time
Wayve
Wayve: British autonomous-driving software licensing vehicle-agnostic AI Driver technology to automakers and fleet owners.
10+ YOE10+ years building large-scale distributed systems or ML infrastructure, 3+ years at staff/principal level, experience with Spark, Ray, Kubernetes, Airflow, MLflow, reliability/observability, mentoring, and optimization or scheduling systems.
Spark, Ray, Kubernetes, Airflow, MLflow, React, Flask, FastAPI
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Machine Learning Engineer
Glendale or San Francisco or New York
$142k-$199k/yr HybridFull Time
ESPN
ESPN: -controlled American sports media serving fans through television networks, digital platforms, streaming, radio, and live events.
5+ YOEExperience building and operating large-scale data/ML systems, strong distributed systems fundamentals, streaming/batch technologies (Kafka, Kinesis, Spark, Flink), Python, plus working knowledge of Java/Scala/Go/C++, cloud-native (AWS, containers, Kubernetes, IaC), observability, and collaboration skills; Bachelor's or Master's required.
Kafka, Kinesis, Spark, Flink, Python, Java, Scala, Go, C++, AWS, Kubernetes, containers, IaC tools
Job PostingView all

Explore Jobs

Expand Your Job Search