79 pyspark data engineer jobs at 45 companies in Berkeley, IL
2d
Save
Mark Applied
Hide
2d
Data Engineer
Durham or Boston or Princeton or Emeryville or Evanston or Evanston or New York City or Rockville
$85k-$105k/yrHybridFull Time
Beghou Consulting: Strategic consulting and data analytics for life sciences companies.
3+ YOEBachelor's degree,3+ years data engineering experience,proficient with Python/PySpark,Databricks,AWS/Azure,relational databases,Git,CI/CD,and cloud security/IAM.
HAVI: Optimizing global supply chains for leading foodservice and retail brands.
3+ YOE3+ years data engineering with Spark on cloud, hands-on Azure Databricks (PySpark/Scala, Spark SQL, Delta Lake), Azure Data Factory, strong SQL, data quality tooling (Great Expectations/Deequ), IaC (Terraform/Bicep), CI/CD and DevOps for data.
fairlifeNYSE: KO: Produces nutrient-dense and ultra-filtered dairy products and shakes.
5+ YOEBachelor's degree and 5+ years data engineering experience, expert SQL and strong Python, Spark/PySpark, Azure data services, Azure DevOps, dimensional modeling, and manufacturing data experience preferred.
SQL, Python, Spark, PySpark, Azure SQL databases, Event Hubs, Functions, Data Factory, Key Vault, Microsoft Entra, Azure DevOps, Git, YAML, Delta Lake, Fabric, Databricks, KQL, Eventhouse, Application Insights, MCP, IIoT, MES
iManage: Intelligent document and email management software for professionals.
5+ YOEBachelor's degree in CS or equivalent, 5+ years data engineering, hands-on Databricks/Delta Lake/PySpark, Terraform, CI/CD (GitHub Actions, DABs), data modeling, mentoring experience, and experience migrating platforms preferred.
PySpark, Databricks, Azure Databricks, Delta Lake, Databricks Asset Bundles (DABs), Databricks MCP connector, Terraform, GitHub Actions, Microsoft Synapse, Microsoft Power BI, Salesforce, Zendesk, Microsoft Event Hub, Apache Kafka, Informatica, Collibra
Boston or Atlanta or Charlotte or Chicago or Dallas or Houston or Los Angeles or New York
$135k-$195k/yrRemoteFull Time
Paul Hastings: International law firm providing legal and regulatory consulting services.
7+ YOE7+ years data engineering or BI development; 3+ years Azure cloud; 2+ years Azure DevOps; Bachelor's in CS/Engineering; SQL/ETL; PySpark; familiarity with Power BI; professional services experience preferred.
Azure Data Factory, Azure Data Lake Storage, Azure Databricks, Azure DevOps, SQL, PySpark, Git, Power BI, Microsoft Purview
Conagra BrandsNYSE: CAG: Manufactures and markets a variety of packaged food brands.
3+ YOEBachelor's degree and 3+ years building data pipelines with Databricks (SQL and pyspark) and Snowflake; experience with relational/cloud DBs, data lake/warehouse concepts, Microsoft Azure components, and self-service analytics.
Databricks, SQL, pyspark, Snowflake, SQL Server, Oracle ADW, Redshift, Microsoft Azure, ADLS, Microsoft Azure Key Vault, Microsoft Azure Dev Ops, Power BI, Sigma, Microsoft Azure Data Factory, Informatica
Blue Cross Blue Shield Association: National association coordinating independent Blue Cross Blue Shield health plans.
5+ YOE5+ years data engineering experience building cloud-based pipelines for ML/GenAI; Bachelor's/Master's in Computer Science/Engineering; proficiency with PySpark, Databricks, AWS Glue, EMR, Snowflake, Airflow, Kubernetes, Python, and SQL; HIPAA/SOC2/GDPR compliance familiarity.
Munich or New York City or Denver or Chicago or London or Heidelberg or Singapore
OnsiteFull Time
CEPRES: Digital investment analytics platform for private capital markets.
2+ YOE2+ years data engineering experience; strong Python, Pyspark and SQL skills; ETL/ELT and data pipeline architecture; exposure to cloud (AWS/Azure/GCP), Git and CI/CD; good communication and problem-solving.
Santa Barbara or Atlanta or Chicago or Agoura Hills
HybridFull Time
Publicis GroupeEuronext Paris: PUB: Global advertising and digital transformation agency holding.
3+ YOE3+ years data engineering experience with Python, SQL, cloud data platforms (Databricks,AWS,Snowflake,Google Cloud); familiarity with AI/automation tools and modern lakehouse architectures.
Python, PySpark, SQL, Spark, Claude Code, Codex, GitHub Copilot, Databricks Genie, Databricks, AWS, Snowflake, Google Cloud, SFTP, APIs
Santa Barbara or Atlanta or Chicago or Agoura Hills
HybridFull Time
Publicis GroupeEuronext Paris: PUB: Global communications, advertising, and digital transformation holding.
3+ YOE3–5 years in data/analytics engineering, strong SQL and Python, experience with Databricks/AWS/Snowflake/Google Cloud, familiarity with AI-assisted development and data ingestion frameworks.
Databricks, AWS, Snowflake, Google Cloud, GitHub Copilot, Claude Code, Codex, Databricks Genie, Python, PySpark, SQL, Spark, SFTP, APIs, CI/CD
84.51°NYSE: KR: Provides retail data analytics and targeted advertising services.
3+ YOE3+ years data development experience with Databricks, PySpark/Spark, Python or Java, SQL/JSON, ETL and data warehousing knowledge; experience with CI/CD, pipeline orchestration, version control, and Agile.
The HartfordNYSE: HIG: Provides business and personal insurance, group benefits, and investments.
8+ YOE8+ years data engineering experience; hands-on with Snowflake, Oracle, Informatica, GitHub, Qlik, SharePlex, DMS; AWS/Azure, EMR, S3, Lambda; Python/PySpark, SQL/NoSQL, CI/CD, ETL; authorized to work in the US without sponsorship.
Schaumburg or Minneapolis or Washington or Springfield or United States
$92k-$164k/yrRemoteFull Time
UnitedHealth GroupNYSE: UNH: Provides health insurance and technology-enabled health care services.
5+ YOEHands-on experience with Azure Data Factory, Databricks/PySpark, Synapse/Fabric, T-SQL and Python; strong data warehouse, ETL, CI/CD, monitoring and migration skills; willing to travel ~10%.
Azure Data Factory, Azure Synapse Analytics, Databricks, Microsoft Fabric, Azure Data Lake Storage Gen2, Delta Lake, Parquet, PySpark, T-SQL, Spark SQL, KQL, PowerShell, Bash, Azure Key Vault, Azure AD, Entra ID, Azure Monitor, Log Analytics, Azure DevOps, ARM, Bicep, Terraform, Synapse Dedicated SQL Pool, Azure SQL, Teradata, Snowflake, Informatica PowerCenter, Unity Catalog, OneLake, pandas, pytest
Cohort AI: A technology enabling data-driven decisions.
Bachelor’s degree in CS or related field; strong SQL; Python/PySpark; experience with Databricks, Snowflake, or BigQuery; remote work experience; US-based preferred.
Komodo Health: Provides AI-driven healthcare data and patient journey analytics software.
Design, implement and scale data pipelines and foundational data systems; experience with Spark, Python, Rust, SQL; leadership experience; healthcare data knowledge beneficial.
Vizient: Provides performance improvement and supply chain services to hospitals
5+ YOE5+ years data engineering experience; strong Azure Databricks, PySpark/Spark SQL, Delta Lake, Azure Data Factory, dbt, SQL, and CI/CD experience; Azure DevOps and Git; knowledge of metadata, governance, Unity Catalog, and enterprise data products. US work authorization required.
Vizient: Healthcare performance improvement and group purchasing organization.
5+ YOE5+ years data engineering experience; strong experience with Azure Databricks, PySpark/Spark SQL, Delta Lake, Azure Data Factory, dbt, Unity Catalog, Azure DevOps/Git and CI/CD; experience with metadata, governance, and enterprise-scale analytical platforms; US work authorization required.
Azure Databricks, PySpark, Spark SQL, Delta Lake, Azure Data Factory, dbt, Unity Catalog, Starburst/Trino, Databricks Asset Bundles, Delta Live Tables, Spark, Databricks notebooks, Azure DevOps, Git, CI/CD, Infrastructure as Code (IaC), Pulumi, Apache Iceberg, AKS/Kubernetes, SQL
Arthur J. Gallagher & Co.NYSE: AJG: Global insurance brokerage and risk management services provider.
7+ YOE3+ MgmtLead data engineering teams; 7+ yrs data engineering; 3+ yrs leading teams; BS in Information Technology; Snowflake, Azure Data Factory, ADLS Gen2, Databricks, Spark, Synapse, SQL Server; scripting (PowerShell, Python, Scala, Java, XML); Data Lake/ETL; CI/CD; agile; strong communication.
Snowflake, Azure Data Factory, ADLS Gen 2, Logic Apps, Azure Functions, Databricks, Apache Spark, Scala, Synapse, SQL Server, PowerShell, Python, Java, XML, Data Lake, Azure Data Lake Storage, CI/CD, Infrastructure as Code
Tata Consultancy ServicesNational Stock Exchange of India: TCS: Global provider of IT services, consulting, and business solutions.
8+ YOE8+ years experience building ETL/ELT pipelines with SQL, Python, PySpark, Azure Data Factory, Databricks and Pulumi; Bachelor of Computer Science required.
ETL/ELT, SQL, Python, PySpark, Azure Data Factory, Databricks, Delta Lake, Azure, ADLS Gen2, Azure SQL, Pulumi, Azure API Services, Azure Function Apps, CI/CD