HiringCafe
Join our community

477 spark engineer jobs at 206 companies in San Rafael, CA

3mo
Save
Mark Applied
Hide
Spark Engineer
Sunnyvale, California, United States
OnsiteFull Time
IT Convergence
IT Convergence: Global provider of Oracle cloud, managed services, and digital transformation.
5+ YOE5+ years in software development; data engineering with pipelines; Python/Perl/Shell; Java; Oracle/MySQL; real-time pipelines with Kafka/Storm.
Python, Perl, Shell, Java, Oracle, MySQL, Kafka, Storm, HDFS, Hive, Pig, Oozie, HBase, Spark
Job PostingView all
2w
Save
Mark Applied
Hide
Software Engineer, Spark Platform
San Francisco or Seattle or Sunnyvale or New York City
$131k-$285k/yr HybridFull Time
DoorDash
DoorDashNYSE: DASH: Local food delivery and on-demand logistics platform.
24+ YOERequires a CS degree or equivalent, 24+ years operating production distributed systems, Spark at scale, Kubernetes, cloud infrastructure, observability, and professional Python, Go, Scala, or Java with SQL fluency.
Apache Spark, Kubernetes, YuniKorn, Volcano, Kueue, Prometheus, OpenTelemetry, AWS, Python, Go, Scala, Java, SQL, Databricks, Amazon EMR
Job PostingView all
1mo
Save
Mark Applied
Hide
Software Engineer, Spark Platform
San Francisco or Sunnyvale or Seattle or New York City
$131k-$285k/yr HybridFull Time
DoorDash
DoorDashNASDAQ: DASH: On-demand delivery platform connecting consumers with local merchants.
24+ YOEExperience operating production distributed systems and Apache Spark at scale, Kubernetes production experience, familiarity with schedulers and observability stacks, cloud (AWS) experience, programming in Python/Go/Scala/Java, and SQL fluency. BS/MS/PhD in CS or equivalent.
Apache Spark, Amazon EMR, Databricks, Kubernetes, YuniKorn, Volcano, Kueue, Spark-on-Kubernetes operator, Prometheus, OpenTelemetry, Python, Go, Scala, Java, SQL, VPC
Job PostingView all
1w
Save
Mark Applied
Hide
Software Engineer - Infrastructure (Senior+)
San Francisco or New York
$100k-$300k/yr OnsiteFull Time
Cogent Security
Cogent Security: Autonomous AI agents for enterprise vulnerability remediation.
5+ YOE5+ years infrastructure engineering experience, hands-on AWS, Clickhouse, Postgres, Spark; startup experience; security and on-prem deployment experience; able to drive large cross-stakeholder projects.
AWS, Clickhouse, Clickhouse Cloud, Postgres, Spark, GCP, Azure, Envoy, Istio, Hashicorp Vault, MicroVM (Runloop), SPIFFE, SPIRE
Job PostingView all
2mo
Save
Mark Applied
Hide
Staff Fullstack Engineer, Agentic Applications
Mountain View, California, United States
$192k-$260k/yr OnsiteFull Time
Databricks
Databricks: A unified platform for data analytics and artificial intelligence.
8+ YOE8+ years software engineering with 2+ years building production LLM/agentic applications; deep Python fluency; experience with agentic frameworks, REST/GraphQL integrations, Databricks/Spark, and technical leadership.
Python, LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel, RAG pipelines, REST, GraphQL, Databricks Unity Catalog, Databricks, Spark, Workday, Greenhouse, ADP, MLflow
Job PostingView all
2w
Save
Mark Applied
Hide
Senior Data Platform Engineer
Austin or San Francisco or New York City or Seattle or Los Angeles or Chicago or Gurugram or Bengaluru
$170k-$230k/yr RemoteFull Time
SentiLink
SentiLink: Provides identity verification and fraud prevention for financial institutions.
5+ YOE5+ years engineering experience; strong Python or Golang skills; building ETL/ELT pipelines at scale with Spark/Hadoop/Kafka; cloud (AWS/Azure/GCP) and database expertise; containerization and IaC experience.
Python, Golang, PostgreSQL (RDS), OpenSearch, Redshift, EMR, Spark, Docker, Lambda, AWS, EKS, SQS, SNS, S3, Kubernetes, Hadoop, Kafka, Microsoft Azure, GCP
Job PostingView all
1mo
Save
Mark Applied
Hide
Principal Engineer, Model Development Platform
Sunnyvale, California, United States
$296k-$335k/yr HybridFull Time
Wayve
Wayve: Develops end-to-end artificial intelligence for autonomous driving systems.
10+ YOE10+ years building large-scale distributed systems or ML infrastructure, 3+ years at staff/principal level, experience with Spark, Ray, Kubernetes, Airflow, MLflow, web frameworks, reliability engineering, and mentoring engineers.
Spark, Ray, Kubernetes, Airflow, MLflow, React, Flask, FastAPI
Job PostingView all
3w
Save
Mark Applied
Hide
Senior Data Engineer
Los Angeles or San Francisco
$170k-$200k/yr HybridFull Time
Tatari
Tatari: A platform for buying and measuring TV advertising campaigns.
5+ YOE5+ years building and operating production ETL pipelines, strong Python and SQL skills, Spark/PySpark, Databricks/Delta Lake, Airflow, data modeling and data quality expertise.
Python, SQL, Spark, PySpark, Databricks, Delta Lake, Airflow, ClickHouse
Job PostingView all
3w
Save
Mark Applied
Hide
Staff Data Engineer
Denver or San Francisco
$190k-$264k/yr HybridFull Time
Checkr
Checkr: AI-powered platform for background checks and identity verification.
10+ YOE10+ years building scalable data platforms; expert PySpark, Python, SQL; experience with Kafka, Spark, Iceberg, data lakes, AWS; strong data modeling and security awareness.
PySpark, Python, SQL, MongoDB, Kafka, Spark, Iceberg, EKS, EMR, Serverless, Glue, Athena, S3, Databricks, Snowflake
Job PostingView all
3mo
Save
Mark Applied
Hide
Staff Machine Learning Platform Engineer
Kitchener-Waterloo or Toronto or San Francisco
$216k-$297k/yr HybridFull Time
Faire
Faire: Online wholesale marketplace connecting independent retailers with unique brands.
8+ YOE8+ years in production ML or data platforms; graduate degree in CS/Engineering/Statistics or related field; strong Databricks/Spark/Delta Lake/MLflow; Python/SQL; cloud and IaC; MLOps.
Databricks, Spark, Delta Lake, MLflow, Terraform, GitHub Actions, Kubernetes, AWS, S3, Python, SQL, Kotlin, Airflow, Snowflake, Unity Catalog
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
San Francisco or United States
$190k-$220k/yr HybridFull Time
Arbital Health
Arbital Health: Provides infrastructure for adjudicating healthcare value-based care contracts.
5+ YOE5+ years building data-intensive SaaS platforms, deep Spark and distributed processing expertise, strong SQL and data modeling, experience with Airflow, Terraform, Databricks, dbt/Great Expectations, healthcare data and HIPAA, Python proficiency.
Python, Spark, SQL, R, Databricks, AWS, Airflow, Terraform, dbt, Great Expectations, Cursor, Claude, Copilot, Gemini, Databricks Genie, Next.js, React, TypeScript, Tailwind CSS, GitHub, Jira, Confluence, Kafka, Structured Streaming, CI/CD
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Data Engineer
San Mateo or New York
$140k-$160k/yr HybridFull Time
Chartmetric
Chartmetric: Provides a global music data analytics platform for professionals.
6+ YOE6+ years building production data systems; expert Python, PostgreSQL, Apache Airflow, Clickhouse, Snowflake, Elasticsearch, Apache Spark, and AWS; bachelor's in CS/Data Engineering preferred or equivalent experience.
Python, PostgreSQL, Apache Airflow, Clickhouse, Snowflake, Elasticsearch, Apache Spark, AWS RDS, AWS
Job PostingView all
2w
Save
Mark Applied
Hide
Hardware Analytics Engineer
Sunnyvale, California, United States
$214k-$225k/yr HybridFull Time
Cerebras Systems
Cerebras SystemsNasdaq: CBRS: Cerebras Systems manufactures specialized computer chips designed for AI.
3+ YOEMaster's degree in EE/CE/CS or related +3 years experience; expertise in large-scale data pipelines, Hive/Spark, Python/SQL, Tableau, Linux, ML for hardware reliability and performance; strong statistical and A/B testing skills.
Python, SQL, Tableau, Hive, Spark, Linux
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Big Data Engineer
San Francisco or New York City or Seattle or Little Rock
$130k-$197k/yr OnsiteFull Time
LiveRamp
LiveRampNew York Stock Exchange: RAMP: Provides a platform for secure data collaboration and identity.
7+ YOE7+ years in software/data engineering; hands-on with Apache Spark, Airflow or Temporal, Dataproc, Kubernetes, Terraform, streaming (Redpanda/Kafka); on-call experience; bachelor’s in CS/Engineering/Math or equivalent.
Apache Spark, Airflow, Temporal, Dataproc, Redpanda, Kafka, Kubernetes, Terraform, Grafana, Linux, Python, Go, Java, Scala, GIT, Subversion, AWS, GCP, Azure
Job PostingView all
4d
Save
Mark Applied
Hide
Practice Customer Engineer, Data Analytics, Google Cloud
Sunnyvale or Atlanta or Austin or Boulder or Chicago or Kirkland or Los Angeles or New York City or Seattle or San Francisco or Portland or Phoenix or Playa Vista
$192k-$267k/yr OnsiteFull Time
Google
GoogleNASDAQ: GOOGL: Provides online search, advertising, cloud computing, and consumer electronics.
10+ YOEBachelor's degree or equivalent, 10 years in cloud-native architecture, Big Data expertise, technical stakeholder engagement, and programming experience with Python, Spark, or SQL.
Google Cloud, Python, Spark, SQL, Informatica, Talend, Alooma, SAP, Data Services, Java, JavaScript, C++, Scala, R, Go, DNS, TCP, Linux, Microsoft Windows, VPN, VPC
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Staff Safety Analytics and Metrics Engineer - Robot Safety
San Carlos, California, United States
$170k-$259k/yr OnsiteFull Time
1X
1X: Manufacturing safe, general-purpose humanoid robots for home and work.
10+ YOE10+ years experience in safety analytics, robotics, autonomy, or data engineering; proficiency with large-scale data infrastructure; Python, SQL, Spark; bachelor’s degree in a technical field required.
Python, SQL, Spark
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Machine Learning Engineer – GeoAI Platform
San Francisco or Bellevue
$185k-$275k/yr HybridFull Time
Wherobots
Wherobots: Cloud-native platform for geospatial data analytics and AI.
5+ YOE5+ years building distributed data or ML systems; experience with Ray/Spark/Dask, GPU inference, Python and scientific stack; geospatial and object-storage expertise.
Ray, Spark, Dask, Python, PyTorch, PyArrow, NumPy, Xarray, Zarr, Cloud-Optimized GeoTIFF (COG), GeoParquet, Parquet, S3, CUDA
Job PostingView all
2mo
Save
Mark Applied
Hide
Staff Software Engineer - Events Platform
New York City or Seattle or San Francisco
$189k-$331k/yr OnsiteFull Time
Rippling
Rippling: Unified platform managing workforce HR, IT, and finance operations
8+ YOE8+ years backend engineering experience, leadership on cross-functional projects, expertise building distributed services, experience with data processing frameworks (e.g., Apache Spark or Flink), AWS and observability experience preferred.
Apache Spark, Flink, AWS, Slack, Microsoft 365
Job PostingView all
1mo
Save
Mark Applied
Hide
Principal Engineer, Model Development Platform
Sunnyvale, California, United States
$296k-$335k/yr HybridFull Time
Wayve
Wayve: Develops AI software for autonomous vehicle navigation.
10+ YOE10+ years building large-scale distributed systems or ML infrastructure, 3+ years at staff/principal level, experience with Spark, Ray, Kubernetes, Airflow, MLflow, reliability/observability, mentoring, and optimization or scheduling systems.
Spark, Ray, Kubernetes, Airflow, MLflow, React, Flask, FastAPI
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Machine Learning Engineer
Glendale or San Francisco or New York
$142k-$199k/yr HybridFull Time
The Walt Disney Company
The Walt Disney CompanyNYSE: DIS: Produces movies, operates theme parks, and provides streaming services.
5+ YOEExperience building and operating large-scale data/ML systems, strong distributed systems fundamentals, streaming/batch technologies (Kafka, Kinesis, Spark, Flink), Python, plus working knowledge of Java/Scala/Go/C++, cloud-native (AWS, containers, Kubernetes, IaC), observability, and collaboration skills; Bachelor's or Master's required.
Kafka, Kinesis, Spark, Flink, Python, Java, Scala, Go, C++, AWS, Kubernetes, containers, IaC tools
Job PostingView all

Explore Jobs

Expand Your Job Search