HiringCafe
Join our community

607 spark engineer jobs at 212 companies in Berkeley, CA

2mo
Save
Mark Applied
Hide
Spark Engineer
Sunnyvale, California, United States
OnsiteFull Time
IT Convergence
IT Convergence: Global provider of Oracle cloud, managed services, and digital transformation.
5+ YOE5+ years in software development; data engineering with pipelines; Python/Perl/Shell; Java; Oracle/MySQL; real-time pipelines with Kafka/Storm.
Python, Perl, Shell, Java, Oracle, MySQL, Kafka, Storm, HDFS, Hive, Pig, Oozie, HBase, Spark
Job PostingView all
1mo
Save
Mark Applied
Hide
Principal Software Engineer - Spark
San Jose or Austin
$270k-$320k/yr HybridFull Time
Cloudera
Cloudera: Provides hybrid data platforms for enterprise analytics and AI.
10+ YOE10+ years in software engineering; strong in Spark, Java/Scala/Python; distributed systems; cloud-native; open-source experience; excellent communication.
Apache Spark, Apache Airflow, Iceberg, Kubernetes, Docker, OpenShift, Rancher, Java, Scala, Python, Go, SQL
Job PostingView all
1mo
Save
Mark Applied
Hide
Principal Machine Learning Engineer, Accelerated Apache Spark
Santa Clara, California, United States
$272k-$431k/yr OnsiteFull Time
NVIDIA
NVIDIANASDAQ: NVDA: Designs GPU-accelerated computing and artificial intelligence hardware.
12+ YOESenior ML/DS engineer with 12+ years of experience; 5+ years leading ML model development; strong Python and ML tooling; experience with GPU-accelerated Spark; expertise in ML methods and feature engineering.
Python, numpy, pandas, scikit-learn, scipy, pytorch, tensorflow, Apache Spark, Scala, Java, C++, XGBoost
Job PostingView all
2d
Save
Mark Applied
Hide
Software Engineer – Presto / Spark Development
San Jose or Austin
$131k-$245k/yr HybridFull Time
IBM
IBMNew York Stock Exchange: IBM: Global technology providing enterprise software, cloud, and consulting.
6+ YOE6+ years software engineering experience with ≥2 years on Presto/Trino or Apache Spark internals; strong Java or Scala; connectors, UDFs/UDAFs, optimizer work; JVM tuning, open table formats, and clear written communication.
Presto, Trino, Apache Spark, Java, Scala, Iceberg, Delta, Hudi, JVM, async-profiler, flamegraphs, Kubernetes, RAPIDS Accelerator, cuDF, Arrow, ORC, Parquet, CI/CD
Job PostingView all
2w
Save
Mark Applied
Hide
Software Engineer, Spark Platform
San Francisco or Sunnyvale or Seattle or New York City
$131k-$285k/yr HybridFull Time
DoorDash
DoorDashNASDAQ: DASH: On-demand delivery platform connecting consumers with local merchants.
24+ YOEExperience operating production distributed systems and Apache Spark at scale, Kubernetes production experience, familiarity with schedulers and observability stacks, cloud (AWS) experience, programming in Python/Go/Scala/Java, and SQL fluency. BS/MS/PhD in CS or equivalent.
Apache Spark, Amazon EMR, Databricks, Kubernetes, YuniKorn, Volcano, Kueue, Spark-on-Kubernetes operator, Prometheus, OpenTelemetry, Python, Go, Scala, Java, SQL, VPC
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Software Engineer I (Data Engineer - Spark, Scala, ETL)
San Francisco, California, United States
RemoteFull Time
UnitedHealth Group
UnitedHealth GroupNYSE: UNH: Provides health insurance and technology-enabled health care services.
6+ YOE6+ years data engineering (Spark/Scala); 4+ years ETL/Databricks; Azure PaaS; Agile; unit tests; SQL/PLSQL; Git; GenAI tools; OO and secure coding.
Apache Spark, Scala, Databricks, Azure, Git, Docker, Splunk, GitHub Actions, SQL, GenAI
Job PostingView all
1mo
Save
Mark Applied
Hide
Principal Machine Learning Engineer, Accelerated Apache Spark
Santa Clara, California, United States
$272k-$431k/yr OnsiteFull Time
NVIDIA
NVIDIANASDAQ: NVDA: Designs graphics processing units and artificial intelligence hardware.
12+ YOEBS/MS/PhD or equivalent; 12+ years ML/DL experience; 5+ years as technical lead; 2+ years with Apache Spark; strong Python and data-science libraries experience; expertise in LLM/GenAI, RL, XGBoost; leadership and deployment experience.
Apache Spark, Python, numpy, pandas, scikit-learn, scipy, PyTorch, TensorFlow, XGBoost, CUDA, Scala, Java, C++
Job PostingView all
2w
Save
Mark Applied
Hide
Lead Software Engineer, Back End (Python, Spark)
New York City or San Francisco
$215k-$246k/yr OnsiteFull Time
Capital One
Capital OneNYSE: COF: Financial services offering credit cards, banking, and loans.
4+ YOEBachelor's degree, 4+ years professional software engineering experience, 1+ year cloud computing (AWS/Azure/GCP), proficiency in Java/Python/SQL/Node/Go/Scala, and experience with containerization and cloud services.
Python, Spark, Java, SQL, Node, Node.js, Go, Scala, Docker, Kubernetes, AWS, Microsoft Azure, Google Cloud
Job PostingView all
1mo
Save
Mark Applied
Hide
Software Development Engineer II, AWS Analytics - Spark Engine
East Palo Alto or Los Angeles County
$165k-$224k/yr OnsiteFull Time
Amazon
AmazonNASDAQ: AMZN: Global online retail and cloud computing technology provider.
3+ YOE3+ years professional software development, 2+ years design/architecture experience, experience with large-scale distributed systems and object-oriented design, Bachelor's in CS/Engineering/Mathematics, programming in C#, C++, Java, or Perl.
EMR, Glue ETL, Athena, EKS, EC2, Serverless, Spark, AI, C#, C++, Java, Perl
Job PostingView all
3mo
Save
Mark Applied
Hide
Senior Data Engineer
Denver or San Francisco or Nashville or Santiago
$157k-$218k/yr OnsiteFull Time
Checkr
Checkr: AI-powered platform for background checks and identity verification.
5+ YOE5+ years engineering experience building scalable data platforms; proficiency in Python, PySpark, SQL; experience with ETL/ELT, batch and stream processing; familiarity with Kafka, Spark, Iceberg, AWS services.
Python, PySpark, SQL, MongoDB, Kafka, Spark, Iceberg, AWS, EKS, EMR, Glue, Athena, S3, Databricks, Snowflake
Job PostingView all
3w
Save
Mark Applied
Hide
Data Platform Engineer
Bengaluru or Novato
OnsiteFull Time
2K
2KNASDAQ: TTWO: Publishes and develops global video game franchises and entertainment.
6+ YOEBachelor's/Master's in CS or engineering, 6+ years in data platform/distributed systems, mastery of SQL and Python, Spark, Snowflake/Databricks, AWS services, Airflow, and data modeling expertise.
Snowflake, Databricks, Kafka, Kinesis, Spark, PySpark, AWS S3, AWS EKS, AWS IAM, AWS SNS, AWS SQS, Apache Iceberg, SQL, Python, Apache Airflow, Terraform, Monte Carlo, DataHub, Soda, Docker, Kubernetes, Pinecone, Weaviate, Milvus, Model Context Protocol (MCP)
Job PostingView all
2mo
Save
Mark Applied
Hide
Sr. Machine Learning Engineer
Sunnyvale, California, United States
$191k-$220k/yr OnsiteFull Time
Illumio
Illumio: Provides zero-trust segmentation software to contain cyberattacks.
5+ YOE5–8 years backend engineering with Java, Python, or Go; distributed systems; Kafka; Spark/Flink; Kubernetes; cloud deployments; agentic frameworks; Langfuse; Evals.
Java, Python, Go, Kafka, Spark, Flink, Kubernetes, Terraform, Langfuse, Evals
Job PostingView all
3w
Save
Mark Applied
Hide
Lead Engineer
Bengaluru or Noida or Santa Clara
HybridFull Time
Eightfold
Eightfold: An AI providing a talent intelligence platform to help organizations hire, develop, and retain employees.
7+ YOE7+ years distributed systems experience; strong programming in Java/Scala/Python/Go; experience with Spark, Kafka, Airflow, Flink, or Snowflake; operating production systems on AWS; technical leadership and mentoring.
Java, Scala, Python, Go, Spark, Kafka, Airflow, Flink, Snowflake, AWS
Job PostingView all
1mo
Save
Mark Applied
Hide
Staff Fullstack Engineer, Agentic Applications
Mountain View, California, United States
$192k-$260k/yr OnsiteFull Time
Databricks
Databricks: A unified platform for data analytics and artificial intelligence.
8+ YOE8+ years software engineering with 2+ years building production LLM/agentic applications; deep Python fluency; experience with agentic frameworks, REST/GraphQL integrations, Databricks/Spark, and technical leadership.
Python, LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel, RAG pipelines, REST, GraphQL, Databricks Unity Catalog, Databricks, Spark, Workday, Greenhouse, ADP, MLflow
Job PostingView all
6d
Save
Mark Applied
Hide
Data Engineer (Multiple Positions)
San Jose, California, United States
$156k-$317k/yr OnsiteFull Time
ByteDance
ByteDance: Developing AI-driven content platforms and mobile applications.
1+ YOEMaster's+1yr or Bachelor's+3yrs in CS/Engineering/Info Systems/Data Science/Statistics; experience designing data warehouses, ETL with Hive/Spark/Hadoop/SQL Server, SQL/Python/Java, and building data services with HBase/Elastic Search/ClickHouse.
Hive, Spark, Hadoop, SQL Server, SQL, Python, Java, HBase, Elastic Search, ClickHouse
Job PostingView all
3w
Save
Mark Applied
Hide
Principal Engineer, Model Development Platform
Sunnyvale, California, United States
$296k-$335k/yr HybridFull Time
Wayve
Wayve: Develops end-to-end artificial intelligence for autonomous driving systems.
10+ YOE10+ years building large-scale distributed systems or ML infrastructure, 3+ years at staff/principal level, experience with Spark, Ray, Kubernetes, Airflow, MLflow, web frameworks, reliability engineering, and mentoring engineers.
Spark, Ray, Kubernetes, Airflow, MLflow, React, Flask, FastAPI
Job PostingView all
1w
Save
Mark Applied
Hide
Senior Software Engineer, Infrastructure, Managed Spark Platform and Foundations
Sunnyvale, California, United States
$174k-$253k/yr OnsiteFull Time
Google
GoogleNASDAQ: GOOGL: Provides online search, advertising, cloud computing, and consumer electronics.
5+ YOEBachelor's degree or equivalent experience,5+ years software development,3+ years building large-scale distributed infrastructure,EMR/Excel not required,experience with distributed computing and API design.
Open Telemetry, JMX, Spark, Ray, Flink, Presto
Job PostingView all
1w
Save
Mark Applied
Hide
Site Reliability Engineer, Compute Platform
San Jose, California, United States
$156k-$388k/yr OnsiteFull Time
TikTok
TikTok: Global short-form video hosting and social media platform.
Bachelor's in CS/Engineering, strong Linux, networking, databases, Kubernetes, SRE/DevOps toolset knowledge, experience with ClickHouse/Spark/Presto/Doris/Hadoop, coding in Python/Shell/Java/Go, strong problem-solving and communication.
ClickHouse, Spark, Presto, Doris, Hadoop, Kubernetes, Python, Shell, Java, Go
Job PostingView all
3w
Save
Mark Applied
Hide
Senior AI Platform Engineer
San Francisco or Boston or Jacksonville or New York or Windsor
$140k-$180k/yr HybridFull Time
SS&C Technologies
SS&C TechnologiesNasdaq: SSNC: Provides software and technology for financial and healthcare sectors.
5+ YOE5+ years Python experience; data engineering/distributed systems (Spark, Hive, Trino, StarRocks, DBT, Airflow, Kubernetes); LLM and agent frameworks (LangChain, LlamaIndex, DSPy, OpenAI MCP, A2A); API design (REST, GraphQL, gRPC); cloud/containerization and enterprise security.
Python, Spark, Hive, Trino, StarRocks, DBT, Airflow, Kubernetes, Docker, LangChain, LlamaIndex, DSPy, OpenAI MCP, A2A, REST, GraphQL, gRPC, pgvector, Pinecone, Weaviate, Milvus, Vespa, Kafka, Pulsar, OpenTelemetry, DataHub, MLflow, Prometheus, Hugging Face
Job PostingView all
3w
Save
Mark Applied
Hide
Senior Data Engineer
San Francisco or United States
$190k-$220k/yr HybridFull Time
Arbital Health
Arbital Health: Provides infrastructure for adjudicating healthcare value-based care contracts.
5+ YOE5+ years building data-intensive SaaS platforms, deep Spark and distributed processing expertise, strong SQL and data modeling, experience with Airflow, Terraform, Databricks, dbt/Great Expectations, healthcare data and HIPAA, Python proficiency.
Python, Spark, SQL, R, Databricks, AWS, Airflow, Terraform, dbt, Great Expectations, Cursor, Claude, Copilot, Gemini, Databricks Genie, Next.js, React, TypeScript, Tailwind CSS, GitHub, Jira, Confluence, Kafka, Structured Streaming, CI/CD
Job PostingView all

Explore Jobs

Expand Your Job Search