HiringCafe
Join our community

417 pyspark developer jobs at 167 companies in California

1mo
Save
Mark Applied
Hide
Sr. Software Engineer - PySpark
Scottsdale or San Francisco or Chicago or New York City
$118k-$152k/yr HybridFull Time
Early Warning Services
Early Warning Services: U.S. bank-owned fintech and consumer reporting agency providing identity, fraud-risk, and real-time payment solutions to financial institutions.
5+ YOEBachelor's-level education typical; minimum 5 years software engineering experience; strong with Git, Spring/Spring Boot, SQL, REST, Docker/Kubernetes, Kafka, automated testing and DevOps practices.
Git, Git flow, Spring, Spring Boot, SQL, Confluence, CI/CD, Docker, Kubernetes, Kafka, Hadoop, Hive, XML, JSON, YAML, Redis, HBASE, AeroSpike, REST
Job PostingView all
2w
Save
Mark Applied
Hide
Senior Developer
Palo Alto, California, United States
$206k-$306k/yr HybridFull Time
SAP
SAPNYSE: SAP: Global leader in enterprise application software and cloud services.
4+ YOEBachelor's degree with 6 years or master's degree with 4 years in software development; experience with Python/PySpark, Java or related languages, databases, cloud platforms, Kubernetes, CI/CD, and distributed data systems.
Python, PySpark, Java, Scala, C, C++, Golang, SAP HANA Cloud, Postgres, MySQL, Amazon Web Services, Microsoft Azure, Google Cloud, Spark, Flink, Hadoop, Kafka, Kubernetes, Helm, GitHub Actions, Jenkins, GitOps, Agile
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Database Engineer
United States or California or Illinois or Montana or South Dakota
$107k-$147k/yr RemoteFull Time
Humana
HumanaNYSE: HUM: Health insurance and care services provider.
3+ YOE3+ years ETL experience; proficiency with SSIS/SSMS, SQL and/or Python/PySpark; Databricks experience or certification; software development and testing experience; strong communication and stakeholder skills; self-starter and critical thinker.
SSIS, SSMS, SQL, Python, PySpark, Databricks, Azure, Power BI, HireVue
Job PostingView all
1w
Save
Mark Applied
Hide
Lead Software Engineer
New York City or Glendale
$163k-$219k/yr HybridFull Time
Disney Streaming
Disney Streaming: Technology subsidiary operating direct-to-consumer streaming services for Disney, Hulu, and ESPN+ audiences.
7+ YOE7+ years in backend development and AI/data applications; Python, Spark/PySpark, scalable APIs, distributed systems, AI/ML, foundation models, cloud deployments, GitHub, Docker, and AWS/EKS experience.
FastAPI, Flask, Spark, PySpark, Python, Claude, GPT-4, OpenAI, Qwen, LangChain, LangGraph, PyTorch, TensorFlow, Cursor, Claude Code, GitHub, Docker, AWS, EKS, Datadog, Grafana, Conviva, Pandas, Databricks, Snowflake
Job PostingView all
1w
Save
Mark Applied
Hide
Databricks Developer
San Jose or United States or Canada
OnsiteFull Time
Tredence
Tredence: Private data science and AI consultancy helping enterprises turn insights into realized business value.
4+ YOEBachelor's or master's degree in a relevant field; 4–8 years in data engineering and analytics; 3+ years with Databricks and PySpark; Agile experience; advanced Python and SQL skills.
Databricks, Databricks Lakehouse Architecture, PySpark, Python, SQL, Delta Lake, Unity Catalog, Azure Databricks, Azure Data Lake Storage (ADLS), Azure Data Factory, Azure Synapse Analytics, Azure DevOps, GitHub Actions
Job PostingView all
1w
Save
Mark Applied
Hide
Senior Forward Deployment Engineer
New York City or San Francisco or Princeton
RemoteFull Time
Quantiphi
Quantiphi: AI-first digital engineering helping enterprises solve complex business problems with artificial intelligence, cloud, and data engineering.
9+ YOERequires 9+ years of software engineering experience, expertise in full-stack development, data architecture, React, JavaScript/TypeScript, Python, FastAPI, PostgreSQL, Redis, PySpark, AWS, and technical leadership.
React, JavaScript, TypeScript, Python, FastAPI, PostgreSQL, RDS, Aurora, Redis, PySpark, Kestra, AWS, EKS, Kubernetes, GitHub Copilot, Cursor, LLM
Job PostingView all
1w
Save
Mark Applied
Hide
AI Software Engineer (KBase Project)
Berkeley, California, United States
$117k-$146k/yr OnsiteFull Time, Contract
Environmental Genomics and Systems Biology
Environmental Genomics and Systems Biology: A national laboratory conducting scientific research for the U.S. Department of Energy.
5+ YOEBachelor's degree or equivalent and 5+ years in AI agent frameworks, data engineering, and scientific software development; proficiency in Python, LangChain, LangGraph, CrewAI, PySpark, and Delta Lake.
Python, LangChain, LangGraph, CrewAI, KBase, KBase Datastore, Delta Lake, PySpark, PyTorch, TensorFlow, Scikit-learn, GitHub, Agile
Job PostingView all
4d
Save
Mark Applied
Hide
Senior Software Engineer
Sunnyvale, California, United States
$188k-$234k/yr OnsiteFull Time
Walmart
WalmartNYSE: WMT: Global retail, e-commerce, and supply chain.
1+ YOEMaster's degree and 1 year, or bachelor's degree and 3 years, in software engineering or a related field. Experience with Airflow, PySpark, Python, Java, Kafka, React.js, TypeScript, GCP, databases, Docker, and Kubernetes.
Apache Airflow, PySpark, Python, Java, JMeter, Apache Kafka, React.js, TypeScript, Google Cloud Platform (GCP), Cloud Storage (GCS), Dataproc, BigQuery, Azure SQL Database, Elasticsearch, OpenSearch, Docker, Kubernetes, Apache Hive, SQL
Job PostingView all
1mo
Save
Mark Applied
Hide
AI/ML Engineer
Woodland Hills, California, United States
$70k-$75k/yr OnsiteFull Time
Tata Consultancy Services
Tata Consultancy ServicesBSE: 532540: Global leader in IT services, consulting, and business solutions.
0+ YOEExperience with Python, LLMs, RAG, LangChain/LangGraph, PySpark, Airflow, cloud platforms; BSc in Computer Science preferred; 0–2 years experience acceptable.
Python, SQL, LLMs, LangChain, LangGraph, PySpark, Airflow, Databricks, Snowflake, Azure, AWS
Job PostingView all
5d
Save
Mark Applied
Hide
Staff, Software Engineer
Sunnyvale, California, United States
$143k-$286k/yr OnsiteFull Time
Walmart
WalmartNYSE: WMT: Global retail, e-commerce, and supply chain.
4+ YOEBachelor's degree plus 4 years of software engineering experience, or 6 years of experience. Requires Java, Python, PySpark, Airflow, Hive, Kafka, cloud-native architecture, AI/ML, CI/CD, and system design expertise.
Python, Java, PySpark, Airflow, Hive, Kafka, CI/CD, WCAG 2.2 AA
Job PostingView all
2mo
Save
Mark Applied
Hide
Software Engineer, Engineering Systems
Los Angeles, California, United States
$120k-$150k/yr OnsiteFull Time
Apex
Apex: Private American satellite manufacturer building configurable satellite buses at scale for commercial and government customers.
2+ YOEU.S. Person required. Bachelor's in related field or equivalent. 2+ years Palantir Foundry and 2+ years Python/PySpark or SQL. ERP implementation experience, data integration and profiling, strong verbal and written communication.
Palantir Foundry, Python, PySpark, SQL
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Software Engineer, Data Engineering
San Mateo, California, United States
$243k-$295k/yr HybridFull Time
Roblox
RobloxNYSE: RBLX: Global platform for user-created immersive digital experiences.
8+ YOE3+ Mgmt8+ years building scalable ETL pipelines, expertise in SQL/PySpark/Scala, experience with cloud platforms and ETL orchestration, 3+ years leading data engineering, strong debugging and collaboration skills.
Airflow, Dagster, Luigi, Google Cloud Composer, SQL, PySpark, Scala, AWS, GCP, Azure
Job PostingView all
2w
Save
Mark Applied
Hide
Senior Software Engineer, Data Engineering
London or San Francisco
HybridFull Time
Ripple
Ripple: Enables institutions to move, manage, and tokenize value.
5+ YOERequires 5–8 years of data engineering experience, Databricks, Delta Live Tables, Unity Catalog, Delta Lake, PySpark or Spark SQL, SQL, AWS, Python, GitLab, and Terraform expertise.
Databricks, Delta Live Tables, Spark, Delta Lake, PySpark, Spark SQL, SQL, AWS, Python, GitLab, Terraform, Unity Catalog, Fivetran, LLMs
Job PostingView all
2mo
Save
Mark Applied
Hide
Staff Software Engineer Data - DC Tech Lead
Toronto or Alabama or Arkansas or California or Colorado or Florida or Georgia or Illinois or Kentucky or Massachusetts or Michigan or Montana or Missouri or Nevada or New Jersey or New York or North Carolina or Oregon or Pennsylvania or Texas or Washington or Utah or Virginia or Wisconsin
$148k-$242k/yr RemoteFull Time
Afresh
Afresh: Private AI software helping grocery retailers order, manage, and sell food with less waste.
Design and maintain large-scale ETLs using PySpark and DBT; proficiency in Python, SQL, Databricks or Snowflake; 2+ years technical lead experience; strong communication and mentoring skills.
PySpark, DBT, Python, SQL, Databricks, Snowflake
Job PostingView all
20h
Save
Mark Applied
Hide
Palantir Forward Deployed Senior Engineer
Chicago or Milwaukee or Dallas or Columbus or Kirkland or Cincinnati or New York City or Cleveland or Oklahoma City or Austin or Albany or St. Petersburg or Hartford or Pittsburgh or St. Louis or Miami or Sacramento or Raleigh or Minneapolis or Mountain View or Scottsdale or San Francisco or Morristown or Denver or Boston or Philadelphia or Des Moines or Overland Park or Los Angeles or Charlotte or Walnut Creek or Carmel or Seattle or Houston or Arlington or Atlanta or Redmond or Bentonville or Beaverton or Nashville or Detroit or San Diego
$87k-$264k/yr HybridFull Time
Accenture
AccentureNYSE: ACN: Global professional services delivering 360° value.
5+ YOERequires 5+ years in data engineering, 3+ years with Palantir Foundry and applied AI, 5+ years with cloud platforms and Python, PySpark, or Java, plus a bachelor's degree or equivalent experience.
Palantir Foundry, Palantir Artificial Intelligence Platform (AIP), Palantir Gotham, Python, TypeScript, PySpark, Java, AWS, Microsoft Azure, Google Cloud Platform (GCP), Git, GitHub, GitLab, Jenkins, JavaScript
Job PostingView all
1mo
Save
Mark Applied
Hide
Staff Data Engineer - Quickbooks Capital and Lending
Mountain View, California, United States
$203k-$274k/yr OnsiteFull Time
Intuit
IntuitNASDAQ: INTU: A global financial technology platform powering prosperity.
7+ YOE7+ years data engineering experience with Python, PySpark, Spark, SQL, Data Bricks/DataBricks, AWS, Elastic Search, Kafka; BS/MS in CS or equivalent; production support and on-call experience; SDLC familiarity.
Python, PySpark, Spark, SQL, Data Bricks, DataBricks, AWS, Elastic Search, Kafka
Job PostingView all
2mo
Save
Mark Applied
Hide
Machine Learning Engineer III - Document Intelligence
Pleasanton or Vancouver
$160k-$240k/yr HybridFull Time
Workday
WorkdayNASDAQ: WDAY: Enterprise cloud applications for finance and human resources.
3+ YOE3+ years building and deploying production ML systems, 2+ years Python, experience with deep learning, NLP/LLMs, PyTorch/TensorFlow, PySpark/SQL, cloud-native deployment, and scalable ML pipelines.
PyTorch, TensorFlow, LangChain, LangGraph, Python, PySpark, SQL, Docker, K8s, AWS, GCP, PEFT, DeepEval, RAGAS, DSPy
Job PostingView all
3mo
Save
Mark Applied
Hide
Sr. Data Engineer
El Segundo, California, United States
$140k-$150k/yr OnsiteFull Time
Fabletics
Fabletics: Digitally native active lifestyle and apparel brand.
6+ YOERequires 6+ years of data engineering or data platform development experience, strong SQL, Python, PySpark, Databricks, Snowflake, Airflow, GitHub Actions, Terraform, and data governance expertise.
Databricks, PySpark, SQL, Airflow, GitHub Actions, Terraform, Python, Snowflake, AI, LLMs, Teams, Zoom
Job PostingView all
2mo
Save
Mark Applied
Hide
Data Engineer
New York or Los Angeles or San Francisco
$140k-$175k/yr HybridFull Time
Regard
Regard: Private healthcare AI helping health systems diagnose patients and generate complete clinical documentation.
3+ YOEBS or equivalent, 3+ years data engineering experience, Python and SQL proficiency, experience with PySpark and distributed data processing, AWS experience preferred, LLM-assisted development familiarity, and willingness to participate in on-call support.
S3, Apache Iceberg, EMR, PySpark, Dagster, Kubernetes, ClickHouse, PostgreSQL, FastAPI, Metabase, Python, SQL, AWS, AWS Athena, EHR, OMOP CDM, HIPAA, LLM
Job PostingView all
3w
Save
Mark Applied
Hide
Student Worker - Machine Learning Engineer - Data Mining & VLM
Foster City, California, United States
$30/hr OnsiteContract
Zoox
Zoox: Autonomous mobility developing a fully electric robotaxi fleet.
Currently enrolled in a relevant BS or MS program; strong Python, PySpark, and SQL; ML/VLM evaluation and production LLM/VLM experience; available 40 hours weekly for at least three months.
Python, PySpark, SQL, Scala, Git, Spark, LLM, VLM, RAG
Job PostingView all

Explore Jobs

Expand Your Job Search