HiringCafe
Join our community

77 pyspark data engineer jobs at 52 companies in Waukegan, IL

1mo
Save
Mark Applied
Hide
Data Engineer
Chicago, Illinois, United States
$100k-$115k/yr HybridFull Time
HAVI
HAVI: Optimizing global supply chains for leading foodservice and retail brands.
3+ YOE3+ years data engineering with Spark on cloud, hands-on Azure Databricks (PySpark/Scala, Spark SQL, Delta Lake), Azure Data Factory, strong SQL, data quality tooling (Great Expectations/Deequ), IaC (Terraform/Bicep), CI/CD and DevOps for data.
Spark, Azure Databricks, PySpark, Scala, Spark SQL, Delta Lake, Unity Catalog, Databricks Jobs, Delta Live Tables, Azure Data Factory, ADLS Gen2, Key Vault, SQL, Great Expectations, Deequ, Git, pytest, dbx, Azure DevOps, GitHub Actions, Terraform, Bicep, Azure Monitor, Log Analytics, Photon
Job PostingView all
4w
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$110k-$125k/yr OnsiteFull Time
fairlife
fairlifeNYSE: KO: Produces nutrient-dense and ultra-filtered dairy products and shakes.
5+ YOEBachelor's degree and 5+ years data engineering experience, expert SQL and strong Python, Spark/PySpark, Azure data services, Azure DevOps, dimensional modeling, and manufacturing data experience preferred.
SQL, Python, Spark, PySpark, Azure SQL databases, Event Hubs, Functions, Data Factory, Key Vault, Microsoft Entra, Azure DevOps, Git, YAML, Delta Lake, Fabric, Databricks, KQL, Eventhouse, Application Insights, MCP, IIoT, MES
Job PostingView all
1w
Save
Mark Applied
Hide
Data Engineer - Databricks
Chicago or Indianapolis or Fort Wayne or Columbus or Lansing or Denver or Dallas or Atlanta
RemoteFull Time
Resultant
Resultant: Provides data analytics and technology consulting services to organizations.
2+ YOEBachelor's degree or equivalent, 2+ years of production data engineering experience, Databricks, PySpark, Spark SQL, Delta Lake, SQL, cloud data services, data modeling, and strong client communication skills.
Databricks, PySpark, Spark SQL, Delta Lake, Delta Live Tables (DLT), Auto Loader, Structured Streaming, Databricks Workflows, Airflow, Azure Data Factory, Unity Catalog, MLflow, Photon, Git, Azure DevOps, GitHub Actions, GitLab, Terraform, dbt, Kafka, Event Hubs, Power BI, Tableau, Docker, Kubernetes, SQL Server, Postgres, Oracle, Snowflake, Azure, AWS, GCP
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$120k-$160k/yr HybridFull Time
iManage
iManage: Intelligent document and email management software for professionals.
5+ YOEBachelor's degree in CS or equivalent, 5+ years data engineering, hands-on Databricks/Delta Lake/PySpark, Terraform, CI/CD (GitHub Actions, DABs), data modeling, mentoring experience, and experience migrating platforms preferred.
PySpark, Databricks, Azure Databricks, Delta Lake, Databricks Asset Bundles (DABs), Databricks MCP connector, Terraform, GitHub Actions, Microsoft Synapse, Microsoft Power BI, Salesforce, Zendesk, Microsoft Event Hub, Apache Kafka, Informatica, Collibra
Job PostingView all
3w
Save
Mark Applied
Hide
Databricks Data Engineer
United States or Arlington or Tysons or Washington or New York City or Chicago or Austin or Atlanta or Boston or Boulder
$113k-$188k/yr RemoteFull Time
Guidehouse
Guidehouse: Provides management and technology consulting services to diverse organizations.
3+ YOEBachelor's degree,3+ years data engineering experience,proficiency with Python/PySpark/SQL,Databricks/Spark/Delta Lake experience,knowledge of data pipelines,governance,and cloud platforms.
Databricks, PySpark, Python, SQL, Delta Lake, Unity Catalog, Spark, CI/CD, Terraform, Azure, AWS, GCP
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$82k-$120k/yr HybridFull Time
Conagra Brands
Conagra BrandsNYSE: CAG: Manufactures and markets a variety of packaged food brands.
3+ YOEBachelor's degree and 3+ years building data pipelines with Databricks (SQL and pyspark) and Snowflake; experience with relational/cloud DBs, data lake/warehouse concepts, Microsoft Azure components, and self-service analytics.
Databricks, SQL, pyspark, Snowflake, SQL Server, Oracle ADW, Redshift, Microsoft Azure, ADLS, Microsoft Azure Key Vault, Microsoft Azure Dev Ops, Power BI, Sigma, Microsoft Azure Data Factory, Informatica
Job PostingView all
13h
Save
Mark Applied
Hide
Data Engineer
Chicago, Illinois, United States
$100k-$150k/yr HybridFull Time
Paychex
PaychexNASDAQ: PAYX: Provider of integrated payroll, HR, and benefits outsourcing solutions.
4+ YOEBachelor's degree in computer science, data science, engineering, or equivalent experience; 2–4 years of experience with data engineering, cloud platforms, automation, CI/CD, security, and large-scale data processing.
SQL, Snowflake, Databricks, Apache Spark, Azure Data Factory, Integration Runtime, Data Lake, Azure Machine Learning, Cosmos DB, Python, PySpark, R, Ansible Tower, Microsoft Azure, Azure Resource Manager (ARM), Terraform Enterprise, Azure DevOps, GitHub Actions, Salesforce FSC, Salesforce Data Cloud, Extract Transform Load (ETL), Amazon Web Services, Apache Hadoop
Job PostingView all
2mo
Save
Mark Applied
Hide
Data Engineer (AI/ML)
Chicago, Illinois, United States
$101k-$139k/yr HybridFull Time
Blue Cross Blue Shield Association
Blue Cross Blue Shield Association: National association coordinating independent Blue Cross Blue Shield health plans.
5+ YOE5+ years data engineering experience building cloud-based pipelines for ML/GenAI; Bachelor's/Master's in Computer Science/Engineering; proficiency with PySpark, Databricks, AWS Glue, EMR, Snowflake, Airflow, Kubernetes, Python, and SQL; HIPAA/SOC2/GDPR compliance familiarity.
PySpark, Databricks, AWS Glue, EMR, Snowflake, Airflow, Kubernetes, Amazon Bedrock, Bedrock Agent Core, SageMaker, Python, SQL, AWS, Microsoft Azure, HL7, FHIR
Job PostingView all
2mo
Save
Mark Applied
Hide
(Remote, Milwaukee) Data Engineer
Milwaukee or Wisconsin
$80k-$90k/yr RemoteFull Time
Harris ComputerToronto Stock Exchange: CSU: Provides mission-critical software solutions across public sector, healthcare, and utilities.
4+ YOE4+ years working with Microsoft Fabric or similar; expertise in data modeling, ETL/ELT, and data warehousing; proficiency in Python or Spark/PySpark; experience with Azure services and developing data lakehouse solutions.
Microsoft Fabric, Microsoft Azure Synapse, Databricks, Python, Spark, PySpark, Microsoft Azure, Microsoft Power BI
Job PostingView all
2mo
Save
Mark Applied
Hide
Data Engineer (f/m/x)
Munich or New York City or Denver or Chicago or London or Heidelberg or Singapore
OnsiteFull Time
CEPRES
CEPRES: Digital investment analytics platform for private capital markets.
2+ YOE2+ years data engineering experience; strong Python, Pyspark and SQL skills; ETL/ELT and data pipeline architecture; exposure to cloud (AWS/Azure/GCP), Git and CI/CD; good communication and problem-solving.
Python, Pyspark, SQL, Databricks, Snowflake, SQL Server, AWS, Redshift, S3, Glue, Lambda, Azure DevOps, Azure Repo, Power BI, Docker, Jenkins, Octopus, Git, CI/CD, NoSQL
Job PostingView all
2mo
Save
Mark Applied
Hide
(Remote, Milwaukee) Data Engineer
Milwaukee or Wisconsin
$80k-$90k/yr RemoteFull Time
Harris Computer
Harris Computer: Provides mission-critical vertical market software and service solutions.
4+ YOE4+ years experience with Microsoft Fabric or similar (Azure Synapse, Databricks); expertise in data modeling, ETL/ELT, data warehousing; proficiency with Python/Spark/PySpark; experience with Azure services.
Microsoft Fabric, Azure Synapse, Databricks, Python, Spark, PySpark, Azure, Power BI
Job PostingView all
5d
Save
Mark Applied
Hide
Senior Data Engineer (P1129)
Chicago or Cincinnati or United States
$98k-$169k/yr OnsiteFull Time
84.51°
84.51°NYSE: KR: Provides retail data analytics and targeted advertising services.
3+ YOERequires 3+ years in data development, Databricks, PySpark/Spark, and Python or Java; ETL, data warehousing, orchestration, CI/CD, version control, Agile, AI concepts, and a relevant bachelor's degree.
PySpark, Python, Databricks, FastAPI, Helm, Azure Kubernetes Service (AKS), Spark, Java, Azure, Databricks Delta Tables, Delta Lake, Delta Live Tables, Flask, Kubernetes, Structured Streaming, Kafka, Azure Event Hubs, GitHub, GitHub Actions, Terraform, Tableau, Microsoft Power BI, Cognos
Job PostingView all
1w
Save
Mark Applied
Hide
Data Engineer
Chicago, Illinois, United States
$175k-$225k/yr OnsiteFull Time
IMC
IMC: Global proprietary trading firm and market maker.
5+ YOERequires 5+ years in data engineering, Kafka streaming and Big Data pipeline experience, Java, Python, SQL, Spark or Flink, Kubernetes, Docker, monitoring tools, statistics, and Unix scripting.
Kafka, Hadoop, Dremio, Java, Python, Apache Spark, Apache Flink, SQL, S3, HDFS, Databricks, Iceberg, Kubernetes, Docker, Prometheus, Grafana, Alert Manager, Alerta, OpsGenie, Unix, Bash
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$148k-$164k/yr HybridFull Time
Sidley Austin
Sidley Austin: Global law firm providing comprehensive legal and regulatory counsel.
5+ YOE5+ years data engineering experience building scalable ETL/ELT and streaming pipelines; strong Python, SQL, Apache Spark; experience with cloud data platforms and Databricks; bachelor's in CS/Engineering/Data Science or related.
Azure Databricks, Apache Spark, Delta Lake, Azure Data Lake Storage (ADLS Gen2), Python, SQL, Scala, Databricks SQL, Unity Catalog, Databricks Workflows (Jobs), Databricks Notebooks, Azure DevOps, Azure Blob Storage, Azure Key Vault, Azure Data Factory, ERwin Data Modeler, AWS, GCP, PySpark, Spark SQL, CI/CD
Job PostingView all
6d
Save
Mark Applied
Hide
Senior Data Engineer
San Francisco or Los Angeles or Denver or Austin or Chicago or New York City or Seattle or Toronto or Santa Barbara or San Diego
$127k-$190k/yr RemoteFull Time
Invoca
Invoca: AI platform for conversation intelligence and revenue execution.
5+ YOE5+ years in data or software engineering; advanced Python, SQL, Databricks, Spark, and data modeling; experience with orchestration, streaming, cloud infrastructure, data quality, ML datasets, and privacy; bachelor's degree or equivalent.
Python, PySpark, Pandas, Polars, PyArrow, SQL, MySQL, PostgreSQL, Databricks, Delta Lake, Unity Catalog, Workflows, Jobs/Compute, Snowflake, BigQuery, Apache Spark, Airflow, Dagster, Kafka, Kinesis, Spark Structured Streaming, AWS, S3, IAM, FastAPI, Databricks Model Serving, SageMaker endpoints
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Artificial Intelligence Data Engineer
Irving or Chicago or Edina
$102k-$179k/yr OnsiteFull Time
Vizient: Provides performance improvement and supply chain services to hospitals
5+ YOE5+ years data engineering experience; strong Azure Databricks, PySpark/Spark SQL, Delta Lake, Azure Data Factory, dbt, SQL, and CI/CD experience; Azure DevOps and Git; knowledge of metadata, governance, Unity Catalog, and enterprise data products. US work authorization required.
Azure Databricks, PySpark, Spark, Spark SQL, Delta Lake, Delta Live Tables, Azure Data Factory, dbt, Unity Catalog, Starburst/Trino, Databricks Asset Bundles, Databricks notebooks, Azure DevOps, Git, CI/CD, Pulumi, Apache Iceberg, AKS, Kubernetes, SQL
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Artificial Intelligence Data Engineer
Irving or Edina or Chicago
$102k-$179k/yr OnsiteFull Time
Vizient
Vizient: Healthcare performance improvement and group purchasing organization.
5+ YOE5+ years data engineering experience; strong experience with Azure Databricks, PySpark/Spark SQL, Delta Lake, Azure Data Factory, dbt, Unity Catalog, Azure DevOps/Git and CI/CD; experience with metadata, governance, and enterprise-scale analytical platforms; US work authorization required.
Azure Databricks, PySpark, Spark SQL, Delta Lake, Azure Data Factory, dbt, Unity Catalog, Starburst/Trino, Databricks Asset Bundles, Delta Live Tables, Spark, Databricks notebooks, Azure DevOps, Git, CI/CD, Infrastructure as Code (IaC), Pulumi, Apache Iceberg, AKS/Kubernetes, SQL
Job PostingView all
1w
Save
Mark Applied
Hide
Data Engineer - Materials Discovery Research Institute
Skokie, Illinois, United States
$81k-$112k/yr OnsiteFull Time
UL Research Institutes
UL Research Institutes: Performs independent research to advance global public safety science.
4+ YOEBachelor’s degree or equivalent experience, 4+ years in data engineering or analytics engineering, strong SQL and Python, cloud platforms, data pipelines, machine learning, and data governance experience.
SQL, Python, pandas, NumPy, scikit-learn, PyTorch, TensorFlow, Microsoft Azure, Amazon Web Services (AWS), Google Cloud, Terraform, Docker, Kubernetes, Apache Spark, Azure Databricks, Azure Data Factory
Job PostingView all
3w
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$120k-$130k/yr OnsiteFull Time
Tata Consultancy Services
Tata Consultancy ServicesNational Stock Exchange of India: TCS: Global provider of IT services, consulting, and business solutions.
8+ YOE8+ years experience building ETL/ELT pipelines with advanced SQL, Python, PySpark, Azure Data Factory and Databricks; bachelor in computer science required.
SQL, Python, PySpark, Azure Data Factory, Databricks, Azure, ADLS Gen2, Azure SQL, Pulumi, Azure API Services, Azure Function Apps, Delta Lake
Job PostingView all
2w
Save
Mark Applied
Hide
Staff Engineer - Data Engineering
Chicago or Scottsdale or San Francisco or New York City
$175k-$270k/yr HybridFull Time
Early Warning Services
Early Warning Services: Operates payment and risk solutions for the financial industry.
8+ YOE8+ years building large-scale data platforms, distributed systems, cloud-native data pipelines, EM/ML operationalization, and data warehouse/Big Data solutions; Bachelor's in CS or related.
Python, Spark, PySpark, R, Data Robot, Pub/Sub, Kafka, RabbitMQ, Google Cloud Platform, AWS, Azure, Tensorflow, Sage Maker, Scikit, PyCharm, Cloudera, S3, Scality S3, Aerospike, Elastic Search, IBM Infosphere, Oracle, SQL Server, AppDynamics, Kubernetes
Job PostingView all

Explore Jobs

Expand Your Job Search