HiringCafe
Join our community

67 pyspark data engineer jobs at 37 companies in Compton, CA

PromotedHiringCafe
Founding Machine Learning / AI Search Engineer
Cupertino, CA, US
$160k-$310k/yr On-SiteFull Time
HiringCafe
HiringCafe: Building a 100× better job search engine to take on Indeed and LinkedIn.
Build the ML and AI search behind HiringCafe — ranking, recommenders, retrieval, and LLM agents that surface jobs people would never find on their own.
Python, PyTorch, Elasticsearch, LLMs
Help us #BeatIndeed & #BeatLinkedIn
2mo
Save
Mark Applied
Hide
Sr Data Engineer
Santa Monica or Seattle or New York
$142k-$190k/yr HybridFull Time
The Walt Disney Company
The Walt Disney CompanyNYSE: DIS: Produces media content and operates global theme parks.
5+ YOE5+ years in data engineering; strong SQL, PySpark, Databricks; AWS; Airflow; data modeling; BI dashboards; Bachelor in CS/IS or related.
SQL, PySpark, Databricks, Unity Catalog, Snowflake, AWS, Airflow, Git, CI/CD, Delta Lake, S3, IAM, EC2, Glue, Lambda
Job PostingView all
4w
Save
Mark Applied
Hide
Data Engineer
New York or Los Angeles or San Francisco
$140k-$175k/yr HybridFull Time
Regard
Regard: A that builds AI-powered software to improve clinical documentation and advance patient care.
3+ YOEBS or equivalent, 3+ years data engineering experience, Python and SQL proficiency, experience with PySpark and distributed data processing, AWS experience preferred, LLM-assisted development familiarity, and willingness to participate in on-call support.
S3, Apache Iceberg, EMR, PySpark, Dagster, Kubernetes, ClickHouse, PostgreSQL, FastAPI, Metabase, Python, SQL, AWS, AWS Athena, EHR, OMOP CDM, HIPAA, LLM
Job PostingView all
2mo
Save
Mark Applied
Hide
Sr Data Engineer
Santa Monica or Seattle or New York
$142k-$190k/yr HybridFull Time
The Walt Disney Company
The Walt Disney CompanyNYSE: DIS: Produces movies, operates theme parks, and provides streaming services.
5+ YOE5+ years in data engineering; SQL expert; PySpark; Databricks; AWS; Airflow; Databricks Unity Catalog; Snowflake; strong data modeling and ETL/ELT design.
Databricks, PySpark, SQL, Unity Catalog, Snowflake, Airflow, AWS, S3, IAM, Delta Lake, Git, CI/CD
Job PostingView all
1w
Save
Mark Applied
Hide
Principal, Data Engineer
Irvine or Dallas
$155k-$195k/yr OnsiteFull Time
Cotality
Cotality: Provides property data, analytics, and workflow intelligence solutions.
7+ YOE7+ years IT experience, bachelor's in CS/Engineering or equivalent, expertise in cloud data platforms, Python/PySpark, data modeling, big data technologies, graph DBs, orchestration tools, and enterprise data architecture.
AWS, Google Cloud Platform (GCP), Python, PySpark, Neo4j, Google Spanner, Hadoop, Spark, Elasticsearch, Google Cloud Dataflow, Apache Beam, Apache Airflow, BMAD, Spec Kit, SQL, NoSQL
Job PostingView all
1w
Save
Mark Applied
Hide
Senior Data Engineer
Costa Mesa, California, United States
$166k-$220k/yr OnsiteFull Time
Anduril Industries
Anduril Industries: Defense technology building autonomous military hardware and software.
3+ YOE3+ years in data engineering, strong Python and SQL skills, experience with Spark/PySpark, dbt, SQLMesh, Palantir Foundry, cloud platforms (AWS/Azure/GCP), data orchestration (Flyte), and data formats (Apache Iceberg).
Palantir Foundry, SQLMesh, Athena, Apache Iceberg, Flyte, Python, Spark, PySpark, SQL, dbt, AWS, Azure, GCP, Tableau
Job PostingView all
2mo
Save
Mark Applied
Hide
Lead Data Engineer
Los Angeles, California, United States
$130k-$170k/yr HybridFull Time
EXL
EXLNASDAQ: EXLS: Provides data analytics and digital operations solutions to businesses.
6+ YOE2+ MgmtLead data engineering role requiring 6-8 years of experience, Databricks stack, SQL, Python, PySpark, data warehousing, Airflow, cloud platforms, and leadership responsibilities.
Databricks, SQL, Python, PySpark, Airflow, Cron, AWS, GCP, Azure, Tableau, Power BI, Hadoop, Hive, HBase, Pig, Spark
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Data Engineer
Boston or Atlanta or Charlotte or Chicago or Dallas or Houston or Los Angeles or New York
$135k-$195k/yr RemoteFull Time
Paul Hastings
Paul Hastings: International law firm providing legal and regulatory consulting services.
7+ YOE7+ years data engineering or BI development; 3+ years Azure cloud; 2+ years Azure DevOps; Bachelor's in CS/Engineering; SQL/ETL; PySpark; familiarity with Power BI; professional services experience preferred.
Azure Data Factory, Azure Data Lake Storage, Azure Databricks, Azure DevOps, SQL, PySpark, Git, Power BI, Microsoft Purview
Job PostingView all
5d
Save
Mark Applied
Hide
Principal, Data Engineer
Irvine or Dallas
$155k-$195k/yr OnsiteFull Time
Cotality
Cotality: Provider of property intelligence, data, and analytics solutions.
7+ YOE7+ years IT experience in data architecture; hands-on GCP, Dataflow, BigQuery, Python/PySpark, Airflow; experience with graph and SQL/NoSQL databases; strong communication and leadership skills.
Google Cloud Platform (GCP), Dataflow, BigQuery, Python, PySpark, Neo4j, Google Spanner, Apache Airflow, AWS, Hadoop, Spark, Elasticsearch, Apache Beam, BMAD, Spec Kit, SQL, NoSQL
Job PostingView all
3mo
Save
Mark Applied
Hide
Data Engineer
Pasadena, California, United States
$90k-$150k/yr HybridFull Time
GEI Consultants
GEI Consultants: Provides multi-disciplinary engineering and environmental consulting services.
4+ YOE4+ years data engineering; Python/SQL; Azure data services; Data governance; pipelines; CI/CD; Databricks a plus.
Python, SQL, PySpark, Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, Azure SQL, Azure AI Search, Cosmos DB, Databricks, Delta Lake, Unity Catalog, Terraform, Azure DevOps
Job PostingView all
3mo
Save
Mark Applied
Hide
Data Engineer
Wakefield or Denver or New York or Atlanta or Cary or Dallas or Pasadena or Rancho Cordova or West Palm Beach
$90k-$150k/yr HybridFull Time
GEI Consultants
GEI Consultants: Professional engineering and environmental consulting services.
4+ YOE4+ years of data engineering; Python and SQL; Azure data services; ETL/ELT; data governance; supports AI/RAG and analytics.
Python, SQL, PySpark, Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, Azure SQL, Cosmos DB, Databricks, Delta Lake, Unity Catalog, Power BI, API, Graph connectors, CI/CD, Terraform
Job PostingView all
4d
Save
Mark Applied
Hide
Senior Data Engineer
North Hollywood or New York
$140k-$160k/yr HybridFull Time
Versant
VersantNasdaq: VSNT: Operates cable television networks and digital media entertainment platforms.
5+ YOE5+ years building production data pipelines with Spark-based platforms, expertise in PySpark and SQL, Databricks experience preferred, bachelor's in CS/data engineering or equivalent, strong CI/CD, orchestration, and data governance skills.
Delta Lake, Databricks Auto Loader, Databricks, Structured Streaming, Change Data Capture (CDC), Unity Catalog, Spark, Photon, Databricks Workflows, Delta Live Tables, Airflow, MLflow, Presto, Flink, Git, Dagster, EMR, Dataproc, Lakeflow Spark Declarative Pipelines (SDP), ChatGPT
Job PostingView all
2w
Save
Mark Applied
Hide
Senior Data Engineer
Los Angeles or New York City or United States or India
$185k-$200k/yr HybridFull Time
Fuel Cycle
Fuel Cycle: Cloud platform for market research and customer feedback analysis.
5+ YOE5+ years building production Databricks lakehouses; strong PySpark, Python, SQL; experience with Unity Catalog, Delta Lake, dbt, orchestrators (Dagster/Prefect/Workflows), and AWS data services.
Databricks, Unity Catalog, Delta Live Tables, Databricks SQL, Workflows, Delta Lake, dbt, PySpark, Python, MySQL, PostgreSQL, Dagster, Prefect, Databricks Workflows, Databricks Vector Search, pgvector, Amazon OpenSearch, Amazon Bedrock, AgentCore, Strands, OpenMetadata, S3, IAM, Databricks Asset Bundles, Terraform, Apache Iceberg, AWS Glue, Athena, DynamoDB, SQL
Job PostingView all
3mo
Save
Mark Applied
Hide
Durability Data Analytics Engineer
Irvine, California, United States
$97k-$121k/yr OnsiteFull Time
Rivian
RivianNASDAQ: RIVN: Designs and manufactures electric vehicles and charging networks.
1+ YOEBachelor's degree in Mechanical Engineering, Aerospace Engineering, or Computer Science. 1-3 years data analytics experience in automotive/engineering. Proficiency in Python, PySpark, Matlab, SQL; knowledge of machine learning and fatigue life estimation methods.
Python, PySpark, Matlab, SQL, Pandas, Numpy
Job PostingView all
3w
Save
Mark Applied
Hide
Data Engineer
Irvine, California, United States
$87k-$123k/yr OnsiteFull Time
Edwards Lifesciences
Edwards LifesciencesNYSE: EW: Designs and manufactures medical devices for structural heart disease.
Bachelor's degree in a related field plus hands-on Databricks, Apache Spark, SQL, and Python experience; strong data engineering, CI/CD, cloud, governance, and PHI handling skills.
Databricks Lakehouse Platform, Databricks, Databricks Workflows, Databricks Repos, Databricks Jobs, Apache Spark, PySpark, Scala, SQL, Python, Delta Lake, Great Expectations, Deequ, Azure DevOps, GitHub Actions, GitLab CI, Git, AWS EC2, AWS S3, AWS Lambda, Azure, GCP
Job PostingView all
1mo
Save
Mark Applied
Hide
Sr. Data Engineer
El Segundo, California, United States
$140k-$150k/yr OnsiteFull Time
Fabletics
Fabletics: Fabletics sells activewear through a membership-based subscription model.
6+ YOESenior data engineer with 6+ years in data pipelines, SQL, Python, Databricks; Airflow, Terraform; data governance; retail/e-commerce familiarity a plus.
Databricks, PySpark, SQL, Airflow, GitHub Actions, Terraform, Snowflake, CI/CD
Job PostingView all
3mo
Save
Mark Applied
Hide
Senior Data Engineer
Seattle or Chicago or Los Angeles or New York
$160k-$220k/yr OnsiteFull Time
Metropolis
Metropolis: Computer vision technology for checkout-free parking and retail payments.
5+ YOEBachelor's in CS/Engineering; 5+ years in data engineering; Python; SQL; Snowflake/dbt; AWS; ETL/ELT; on-site 4 days/week.
Python, SQL, Snowflake, dbt, Airflow, MWAA, DMS, EKS, SNS, Glue, AWS, MySQL, MS SQL Server, Postgres, Spark, PySpark, Automic, CI/CD, ETL/ELT
Job PostingView all
2w
Save
Mark Applied
Hide
Data Engineer
Berkeley or Los Angeles
$111k-$166k/yr HybridFull Time
PBK
PBK: Provides architectural and engineering design for specialized facilities.
5+ YOE5+ years building production data pipelines and integrations; strong SQL (PostgreSQL), Python, Microsoft Azure and Microsoft Fabric experience; API and data governance knowledge; Bachelor's degree or equivalent experience.
Salesforce, Deltek Vantagepoint, Autodesk Forma, Microsoft Azure, Microsoft Fabric, Microsoft Fabric Data Pipelines, Microsoft Fabric notebooks, Microsoft Azure Data Factory, Microsoft Azure Logic Apps, Microsoft Azure Functions, Microsoft Azure Event Grid, Microsoft Azure Service Bus, Databricks, Snowflake, dbt, Apache Spark, Spark, Apache Kafka, MuleSoft, Microsoft 365, Microsoft Power BI, GitHub Enterprise, Microsoft Azure DevOps, REST, JSON, SOAP, PostgreSQL, Python, Apache Airflow
Job PostingView all
1mo
Save
Mark Applied
Hide
Staff Data Platform Engineer
Irvine, California, United States
$154k-$204k/yr HybridFull Time
Rivian and Volkswagen Group Technologies
Rivian and Volkswagen Group Technologies: A joint venture developing software, cloud, and connectivity solutions for next-generation electric vehicles.
3+ YOEBachelor's or Master's in CS/Data Science, 3+ years building large-scale data processing systems, Databricks and Terraform experience, cloud (AWS/GCP/Azure) familiarity, PySpark preferred, strong communication and problem-solving.
Databricks, Delta, Spark, Unity Catalog, Terraform, CI/CD, PySpark, Spark SQL, Spark Streaming, AWS, GCP, Azure, IAM, VPC, EC2, S3
Job PostingView all
3mo
Save
Mark Applied
Hide
Senior Data Engineer
Calabasas, California, United States
$123k-$132k/yr OnsiteFull Time
AMH
AMHNYSE: AMH: Leasing and developing single-family rental homes across the United States.
8+ YOESenior data engineer with 8+ years experience; design/build data pipelines and platforms; ML model deployment; leadership and CI/CD.
Azure, AWS, Google Cloud, Databricks, Apache Spark, Kafka, Event Hub, IoT Hub, Power BI, Tableau, Python, CI/CD, RESTful APIs, Azure Data Factory, HDInsight, Cosmos DB, ML Studio, Azure Analysis Services, Cosmo DB, Cosmos DB
Job PostingView all
3d
Save
Mark Applied
Hide
Data Engineer
Irvine, California, United States
$120k-$150k/yr OnsiteFull Time
Tata Consultancy Services
Tata Consultancy ServicesNational Stock Exchange of India: TCS: Global provider of IT services, consulting, and business solutions.
8+ YOESenior data engineer with strong Databricks/Spark, Airflow/Astronomer, data pipeline design, DevOps/CI/CD, and platform management experience.
Databricks, Apache Spark, Airflow, Astronomer, CI/CD
Job PostingView all

Explore Jobs

Expand Your Job Search