HiringCafe
Join our community

175 pyspark data engineer jobs at 81 companies in Berkeley, CA

PromotedHiringCafe
Founding Machine Learning / AI Search Engineer
Cupertino, CA, US
$160k-$310k/yr On-SiteFull Time
HiringCafe
HiringCafe: Building a 100× better job search engine to take on Indeed and LinkedIn.
Build the ML and AI search behind HiringCafe — ranking, recommenders, retrieval, and LLM agents that surface jobs people would never find on their own.
Python, PyTorch, Elasticsearch, LLMs
Help us #BeatIndeed & #BeatLinkedIn
1mo
Save
Mark Applied
Hide
Data Engineer
San Francisco, California, United States
$140k-$230k/yr OnsiteFull Time
ReadyOn
ReadyOn: AI-native labor operating system for frontline enterprise workforces.
5+ YOE5+ years data engineering experience; strong SQL; Python; TypeScript; Airflow; AWS Glue; PySpark; Snowflake/PostgreSQL; backend frameworks; observability and data quality mechanisms.
Python, TypeScript, Apache Airflow, PySpark, AWS Glue, Snowflake, PostgreSQL, NestJS, GraphQL, AWS
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer
New York or Los Angeles or San Francisco
$140k-$175k/yr HybridFull Time
Regard
Regard: A that builds AI-powered software to improve clinical documentation and advance patient care.
3+ YOEBS or equivalent, 3+ years data engineering experience, Python and SQL proficiency, experience with PySpark and distributed data processing, AWS experience preferred, LLM-assisted development familiarity, and willingness to participate in on-call support.
S3, Apache Iceberg, EMR, PySpark, Dagster, Kubernetes, ClickHouse, PostgreSQL, FastAPI, Metabase, Python, SQL, AWS, AWS Athena, EHR, OMOP CDM, HIPAA, LLM
Job PostingView all
1w
Save
Mark Applied
Hide
Sr. Data Engineer
San Francisco, California, United States
$120k-$160k/yr HybridFull Time
SS&C Technologies
SS&C TechnologiesNasdaq: SSNC: Provides software and technology for financial and healthcare sectors.
Experienced in data modeling, large-scale data pipelines, ETL/ELT, Python and PySpark, machine learning/NLP, data warehousing, and collaboration across engineering and product teams.
Python, numpy, pandas, scikit-learn, nltk, TensorFlow, matplotlib, PySpark
Job PostingView all
3w
Save
Mark Applied
Hide
Senior Data Engineer
United States or San Francisco or New York City or Washington or Austin
$118k-$179k/yr RemoteFull Time
Samsara
SamsaraNYSE: IOT: Connected Operations Cloud platform for physical operations and IoT.
8+ YOEBachelor's in CS or equivalent, 8+ years as a software/data engineer, 5+ years building production data pipelines and Spark/PySpark experience, strong Python and SQL, cloud data warehouse and ETL tooling experience.
Spark, PySpark, Databricks, Python, SQL, Snowflake, Google BigQuery, Apache Iceberg, Fivetran, dbt, NetSuite, Salesforce, MySQL, AWS RDS/Aurora, PostgreSQL, Oracle, Microsoft SQL Server, AWS, Azure, GCP, Splunk, Datadog, AWS CloudWatch, API Gateway, Lambda, S3, SNS, SQS, SecretsManager, MCP (Model Context Protocol)
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
United States or San Francisco or New York City
$120k-$235k/yr OnsiteFull Time
Microsoft
MicrosoftNASDAQ: MSFT: Develops software, services, devices, and cloud computing solutions.
3+ YOEMaster's (3+ yrs) or Bachelor's (4+ yrs) in CS/Math/Engineering or equivalent experience; experience building production data systems, instrumentation/telemetry, experimentation, PySpark/SQL, and mentoring engineers; must pass Microsoft background checks.
ADLS, Azure Synapse, Microsoft Fabric, Databricks, PySpark, SQL, Cosmos, Scope, Conform, Aria
Job PostingView all
1mo
Save
Mark Applied
Hide
Staff Data Engineer - Data & ML Platform
San Francisco, California, United States
$198k-$296k/yr HybridFull Time
Hinge Health
Hinge HealthNYSE: HNGE: Digital provider of musculoskeletal care and physical therapy
8+ YOE8+ years data engineering; strong Python and SQL; distributed processing (PySpark/Flink); cloud pipelines; mentoring.
Python, SQL, Flink, PySpark, Kafka, Delta Lake, dbt, Airflow, Aurora PostgreSQL, AWS
Job PostingView all
3mo
Save
Mark Applied
Hide
Senior Data Engineer
Denver or San Francisco or Nashville or Santiago
$157k-$218k/yr OnsiteFull Time
Checkr
Checkr: AI-powered platform for background checks and identity verification.
5+ YOE5+ years engineering experience building scalable data platforms; proficiency in Python, PySpark, SQL; experience with ETL/ELT, batch and stream processing; familiarity with Kafka, Spark, Iceberg, AWS services.
Python, PySpark, SQL, MongoDB, Kafka, Spark, Iceberg, AWS, EKS, EMR, Glue, Athena, S3, Databricks, Snowflake
Job PostingView all
2mo
Save
Mark Applied
Hide
Sr. Data Engineer
Foster City, California, United States
$123k-$191k/yr HybridFull Time
Visa
VisaNYSE: V: Global payment technology facilitating electronic funds transfers.
2+ YOE2+ years experience with a bachelor’s degree or advanced degree; strong Python, SQL, PySpark; Hadoop ecosystem; AWS data services; Airflow; CI/CD; data quality and observability; ability to collaborate with stakeholders.
Python, SQL, PySpark, Hadoop, Spark, Pandas, Airflow, Snowflake, Databricks, AWS, S3, EMR, Glue Catalog, Claude Code
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Science Engineer
San Jose or California or United States
$109k-$201k/yr OnsiteFull Time
Adobe
AdobeNASDAQ: ADBE: Provides software for digital media creation and marketing analytics
8+ YOESenior data engineer with 8+ years experience, distributed data and cloud technologies (Hadoop, Spark, Databricks, S3, EMR), streaming (Kafka/Kinesis), strong SQL and Python/PySpark/Scala skills, CI/CD and orchestration experience, and strong communication.
Hadoop, Hive, Presto, Spark, Databricks, S3, Azure Blob Storage, Notebooks, AWS EMR, Athena, Glue, Delta, Parquet, ORC, Kafka, Kinesis, SQL, Python, PySpark, Scala, Pandas, NumPy, Koalas, APIs, GitHub, Jenkins, Apache Air Flow, Azkaban, AEP, AJO, CJA, Adobe Experience Platform, Adobe Analytics, Customer Journey Analytics, Adobe Journey Optimizer, Collibra, JIRA, Confluence, Copilot, Claude, LLAMA, Databricks Genie, n8n, RAG, MCPs, GenStudio, Adobe Acrobat Studio, Adobe Express, Adobe Firefly, Creative Cloud
Job PostingView all
3mo
Save
Mark Applied
Hide
Staff Data Engineer/Surgical Robotics - OTTAVA
Santa Clara, California, United States
$118k-$179k/yr OnsiteFull Time
Johnson & Johnson
Johnson & JohnsonNYSE: JNJ: Global healthcare providing pharmaceuticals and medical technologies.
7+ YOEBachelor's/Master's in CS/Engineering/Data Science, 7+ years data engineering (5+ for exceptional), Databricks/Spark/Delta Lake, Python/PySpark/SQL, AWS S3, OT integrations (OPC UA, MQTT, MES), data governance and GxP/CSV compliance.
Databricks, Spark, Delta Lake, Databricks Workflows, Python, PySpark, SQL, AWS S3, Kinesis, MSK, OPC UA, MQTT, MES, ISA-95
Job PostingView all
3mo
Save
Mark Applied
Hide
Staff Data Engineer: Robotics & Digital Surgery - OTTAVA
Santa Clara, California, United States
$118k-$179k/yr OnsiteFull Time
Johnson & Johnson
Johnson & JohnsonNYSE: JNJ: Provides pharmaceutical products and medical technology healthcare solutions.
7+ YOE7+ years data engineering; Databricks; AWS; Python; PySpark; ETL/ELT; data governance; OT data; onsite.
Databricks, Spark, Delta Lake, Databricks Workflows, Python, PySpark, SQL, AWS S3, Kinesis, MSK, OPC UA, MQTT, MES
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer
San Francisco or Denver
$170k-$220k/yr HybridFull Time
Windfall
Windfall: Provides consumer financial data and AI-driven go-to-market insights.
4+ YOE4-8 years in data engineering; experience with Apache Beam/Spark/Flink or MapReduce; JVM language proficiency; distributed data processing; experience at a small company; strong communication and ownership.
GCP, Java, Python, Kotlin, BigQuery, PostgreSQL, Scylla/Cassandra, Apache Beam, Apache Spark, Dataflow, Airflow
Job PostingView all
4d
Save
Mark Applied
Hide
Senior, Data Engineer
Bentonville or Sunnyvale
$90k-$234k/yr OnsiteFull Time
Walmart
WalmartNYSE: WMT: Multinational retail operating discount stores and supermarkets.
2+ YOE5+ years data/software engineering experience (or equivalent degree+experience), 2+ years data engineering, expertise with big data technologies, SQL, Scala or Python, Airflow, Kafka, and cloud platforms.
Hadoop, Apache Spark, Apache Hive, GCP, Azure, Scala, Python, Apache Airflow, BigQuery, Spark SQL, Kafka
Job PostingView all
1w
Save
Mark Applied
Hide
Staff Data Engineer
Mountain View or San Diego
$189k-$259k/yr OnsiteFull Time
Intuit
IntuitNASDAQ: INTU: Provides financial software for accounting, tax, and personal finance.
7+ YOE7+ years experience in data analytics/engineering, bachelor's in a quantitative field, proficiency in Java or Python, ETL/data modeling, big data frameworks, and visualization tools; strong communication skills.
Java, Python, Tableau, Qlik, PowerBI, Apache Spark, Hadoop, Databricks
Job PostingView all
3mo
Save
Mark Applied
Hide
Staff Data Engineer: Robotics & Digital Surgery - OTTAVA
Santa Clara, California, United States
$118k-$179k/yr OnsiteFull Time
Johnson & Johnson
Johnson & JohnsonNYSE: JNJ: Develops and manufactures pharmaceutical drugs and medical devices.
7+ YOE7+ years data engineering; Databricks, Spark/Delta Lake, Python/PySpark; AWS S3; ETL/ELT design; data governance; OT data integration; strong communication.
Databricks, Spark, Delta Lake, Databricks Workflows, Python, PySpark, SQL, AWS S3, Kinesis, MSK, OPC UA, MQTT, MES
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
San Mateo or New York City
$140k-$160k/yr HybridFull Time
Chartmetric
Chartmetric: Provides a global music data analytics platform for professionals.
6+ YOE6+ years building production data systems; expert in Python, PostgreSQL, Airflow, Clickhouse, Snowflake, Elasticsearch, Spark, and AWS; Bachelor's in CS/Data Engineering or equivalent experience.
Python, PostgreSQL, Apache Airflow, Clickhouse, Snowflake, Elasticsearch, Apache Spark, AWS, AWS RDS
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Redwood City, California, United States
$185k-$240k/yr HybridFull Time
NimbleRx
NimbleRx: Digital platform for on-demand prescription delivery and pharmacy management.
5+ YOE5+ years building production data pipelines; strong Python (PySpark) and SQL; experience with Spark/EMR, Kinesis, Lambda, Step Functions; Postgres; Iceberg/Trino; Java/Spring; CI/CD and Terraform; AI tooling; cross-team collaboration.
PySpark, SQL, Java, Spring Boot, Kinesis, EMR, S3, Postgres, Iceberg, Trino, Terraform, CI/CD
Job PostingView all
1w
Save
Mark Applied
Hide
Sr. Data Engineer
Palo Alto, California, United States
$145k-$220k/yr RemoteFull Time
Allocate
Allocate: A fintech providing a data-rich platform for discovering, modeling, and managing private market investments.
5+ YOE5+ years data engineering experience; strong AWS (S3, EC2, EKS, Redshift, Glue, Athena) and cloud skills; SQL, Python, data modeling, ETL/ELT, and pipeline production experience; familiarity with vector DBs and ML data workflows.
S3, EC2, ECS, EKS, Athena, Redshift, Glue, Step Functions, Terraform, CloudFormation, SQL, Snowflake, Databricks Delta Lake, Neo4j, AWS Neptune, Python, pandas, PySpark, TypeScript, Node.js, C#, Postgres pgvector, Chroma, Pinecone, LangChain, LlamaIndex, Docker, Kubernetes, Kafka, Kinesis, Airflow, Prefect, dbt
Job PostingView all
3w
Save
Mark Applied
Hide
Data Engineer
Houston or Golden or Reno or Oakland or Salt Lake City
HybridFull Time
Fervo Energy
Fervo Energy: Generates clean electricity using advanced geothermal drilling technology.
2+ YOE2+ years building and operating production data pipelines with Apache Spark, Python/SQL, Databricks, Azure, and Snowflake; experience with streaming, data modeling, data quality, governance, and CI/CD.
Databricks, Delta Lake, Delta Live Tables, Unity Catalog, Databricks Workflows, Databricks SQL, Apache Spark, PySpark, Spark SQL, Structured Streaming, Kafka, Microsoft Event Hubs, Microsoft Azure Data Factory, Microsoft Azure Data Lake Storage (ADLS), Snowflake, Microsoft Power BI, Spotfire, Python, SQL, Git, Azure DevOps, GitHub Actions, dbt, Terraform, Docker, Airflow, Microsoft Entra ID, Microsoft Key Vault, MQTT, OPC UA, SparkplugB, Canary, Ignition, Snowflake Semantic Views, Databricks Metric Views
Job PostingView all
3w
Save
Mark Applied
Hide
AI Data Engineer
Palo Alto or Atlanta or Washington
$160k-$200k/yr HybridFull Time
Quantifind
Quantifind: AI-driven platform for financial risk and fraud detection.
4+ YOEUS citizen with 4+ years' experience, technical education, strong Python and AWS skills, experience with large-scale ETL, Spark/PySpark, PostgreSQL/RDS, Neo4j, and hands-on AI coding tools; must be able to meet in person.
Claude Code, GPT/Codex, Cursor, Windsurf, PostgreSQL, RDS, Python, Spark/PySpark, Scala, AWS, Neo4j
Job PostingView all

Explore Jobs

Expand Your Job Search