HiringCafe
Join our community

104 pyspark data engineer jobs at 64 companies in Illinois

2d
Save
Mark Applied
Hide
Data Engineer
Illinois or Arkansas or Idaho or Georgia or Alabama or Iowa or Utah or California or Arizona or Kansas or Florida or Colorado or Connecticut or Indiana or Delaware or Kentucky
$65k-$173k/yr RemoteFull Time
CVS Health
CVS HealthNYSE: CVS: Provides retail pharmacy, health insurance, and pharmacy benefit management services.
4+ YOERequires 4+ years with SQL and relational databases, 2+ years in cloud and on-premises data engineering platforms, SQL/Python/PySpark, data modeling, warehousing, troubleshooting, and a bachelor's degree in computer science or engineering.
SQL, Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, Databricks, Snowflake, Microsoft SQL Server, Oracle, Teradata, Python, PySpark
Job PostingView all
1w
Save
Mark Applied
Hide
Data Engineer
Durham or Boston or Princeton or Emeryville or Evanston or Evanston or New York City or Rockville
$85k-$105k/yr HybridFull Time
Beghou Consulting
Beghou Consulting: Strategic consulting and data analytics for life sciences companies.
3+ YOEBachelor's degree,3+ years data engineering experience,proficient with Python/PySpark,Databricks,AWS/Azure,relational databases,Git,CI/CD,and cloud security/IAM.
Databricks, Python, PySpark, AWS, Azure, PostgreSQL, Oracle, MySQL, Amazon Redshift, Snowflake, Git, Azure AD, Okta, OAuth, SAML, Azure Data Factory, Informatica, SnapLogic, Boomi, Docker, Kubernetes, AWS ECS, Flask, Django, JavaScript, HTML, CSS, Ajax
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer
Chicago, Illinois, United States
$100k-$115k/yr HybridFull Time
HAVI
HAVI: Optimizing global supply chains for leading foodservice and retail brands.
3+ YOE3+ years data engineering with Spark on cloud, hands-on Azure Databricks (PySpark/Scala, Spark SQL, Delta Lake), Azure Data Factory, strong SQL, data quality tooling (Great Expectations/Deequ), IaC (Terraform/Bicep), CI/CD and DevOps for data.
Spark, Azure Databricks, PySpark, Scala, Spark SQL, Delta Lake, Unity Catalog, Databricks Jobs, Delta Live Tables, Azure Data Factory, ADLS Gen2, Key Vault, SQL, Great Expectations, Deequ, Git, pytest, dbx, Azure DevOps, GitHub Actions, Terraform, Bicep, Azure Monitor, Log Analytics, Photon
Job PostingView all
2w
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$110k-$125k/yr OnsiteFull Time
fairlife
fairlifeNYSE: KO: Produces nutrient-dense and ultra-filtered dairy products and shakes.
5+ YOEBachelor's degree and 5+ years data engineering experience, expert SQL and strong Python, Spark/PySpark, Azure data services, Azure DevOps, dimensional modeling, and manufacturing data experience preferred.
SQL, Python, Spark, PySpark, Azure SQL databases, Event Hubs, Functions, Data Factory, Key Vault, Microsoft Entra, Azure DevOps, Git, YAML, Delta Lake, Fabric, Databricks, KQL, Eventhouse, Application Insights, MCP, IIoT, MES
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$120k-$160k/yr HybridFull Time
iManage
iManage: Intelligent document and email management software for professionals.
5+ YOEBachelor's degree in CS or equivalent, 5+ years data engineering, hands-on Databricks/Delta Lake/PySpark, Terraform, CI/CD (GitHub Actions, DABs), data modeling, mentoring experience, and experience migrating platforms preferred.
PySpark, Databricks, Azure Databricks, Delta Lake, Databricks Asset Bundles (DABs), Databricks MCP connector, Terraform, GitHub Actions, Microsoft Synapse, Microsoft Power BI, Salesforce, Zendesk, Microsoft Event Hub, Apache Kafka, Informatica, Collibra
Job PostingView all
2d
Save
Mark Applied
Hide
Senior Data Engineer
Florida or Montana or Michigan or Massachusetts or Rhode Island or West Virginia or Oregon or Oklahoma or Minnesota or New York or South Carolina or Kentucky or Virginia or Colorado or New Jersey or Wisconsin or Utah or Pennsylvania or Ohio or North Carolina or Nebraska or Mississippi or South Dakota or Maryland or Louisiana or Kansas or Iowa or Illinois or Tennessee or Delaware or Connecticut or California or Hawaii or Arkansas or District of Columbia or Georgia or Missouri or Arizona or Alabama or New Hampshire or Texas or Vermont or Indiana or Washington or Nevada or Idaho or New Mexico or Maine
$143k-$160k/yr RemoteFull Time
AmeriLife
AmeriLife: Distributes insurance, annuities, and retirement solutions for retirees.
7+ YOE7+ years data engineering experience with 3+ years Databricks; strong Spark/PySpark, Delta Lake, SQL, Azure, Git and CI/CD skills; Bachelor's degree required.
Databricks, Apache Spark, Delta Lake, Unity Catalog, Workflows, SQL Warehouses, Delta Live Tables, PySpark, SQL, Azure, Git, dbt, Terraform, Azure DevOps
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer – Insurance Data Platform
Danville, Illinois, United States
$100k-$150k/yr RemoteFull Time
CCMSI
CCMSI: Independent third-party administrator for insurance claims and risk management.
8+ YOE8+ years data engineering experience with 5+ years on insurance data; expert SQL, Python (pandas, PySpark), cloud data warehouse experience, ELT/ETL and orchestration tools, Azure/IaC familiarity, strong communication and stakeholder collaboration skills.
iCE, SQL, Python, pandas, PySpark, SQLAlchemy, Snowflake, Databricks, BigQuery, Synapse, Redshift, Azure, Terraform, Bicep, CloudFormation, Git, CI/CD, Kafka, Kinesis, Event Hubs, Debezium
Job PostingView all
1w
Save
Mark Applied
Hide
Databricks Data Engineer
United States or Arlington or Tysons or Washington or New York City or Chicago or Austin or Atlanta or Boston or Boulder
$113k-$188k/yr RemoteFull Time
Guidehouse
Guidehouse: Provides management and technology consulting services to diverse organizations.
3+ YOEBachelor's degree,3+ years data engineering experience,proficiency with Python/PySpark/SQL,Databricks/Spark/Delta Lake experience,knowledge of data pipelines,governance,and cloud platforms.
Databricks, PySpark, Python, SQL, Delta Lake, Unity Catalog, Spark, CI/CD, Terraform, Azure, AWS, GCP
Job PostingView all
3w
Save
Mark Applied
Hide
Senior Staff Data Engineer
Hartford or Chicago or Columbus or Charlotte
$135k-$203k/yr HybridFull Time
The Hartford
The HartfordNYSE: HIG: Provides property and casualty insurance, group benefits, and wealth management.
8+ YOE8+ years data engineering experience with Snowflake, Oracle, Informatica, GitHub, AWS/Azure, Python/PySpark, EMR, Kafka; certifications in cloud/Snowflake preferred; US work authorization required.
Snowflake, Oracle, Informatica Cloud, PowerCenter, GitHub, Qlik, Shareplex, DMS, Python, PySpark, Spark, Hadoop, EMR, S3, Lambda, AWS, Azure, Kafka, Google Vertex, AWS Bedrock
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$82k-$120k/yr HybridFull Time
Conagra Brands
Conagra BrandsNYSE: CAG: Manufactures and markets a variety of packaged food brands.
3+ YOEBachelor's degree and 3+ years building data pipelines with Databricks (SQL and pyspark) and Snowflake; experience with relational/cloud DBs, data lake/warehouse concepts, Microsoft Azure components, and self-service analytics.
Databricks, SQL, pyspark, Snowflake, SQL Server, Oracle ADW, Redshift, Microsoft Azure, ADLS, Microsoft Azure Key Vault, Microsoft Azure Dev Ops, Power BI, Sigma, Microsoft Azure Data Factory, Informatica
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer (AI/ML)
Chicago, Illinois, United States
$101k-$139k/yr HybridFull Time
Blue Cross Blue Shield Association
Blue Cross Blue Shield Association: National association coordinating independent Blue Cross Blue Shield health plans.
5+ YOE5+ years data engineering experience building cloud-based pipelines for ML/GenAI; Bachelor's/Master's in Computer Science/Engineering; proficiency with PySpark, Databricks, AWS Glue, EMR, Snowflake, Airflow, Kubernetes, Python, and SQL; HIPAA/SOC2/GDPR compliance familiarity.
PySpark, Databricks, AWS Glue, EMR, Snowflake, Airflow, Kubernetes, Amazon Bedrock, Bedrock Agent Core, SageMaker, Python, SQL, AWS, Microsoft Azure, HL7, FHIR
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer (f/m/x)
Munich or New York City or Denver or Chicago or London or Heidelberg or Singapore
OnsiteFull Time
CEPRES
CEPRES: Digital investment analytics platform for private capital markets.
2+ YOE2+ years data engineering experience; strong Python, Pyspark and SQL skills; ETL/ELT and data pipeline architecture; exposure to cloud (AWS/Azure/GCP), Git and CI/CD; good communication and problem-solving.
Python, Pyspark, SQL, Databricks, Snowflake, SQL Server, AWS, Redshift, S3, Glue, Lambda, Azure DevOps, Azure Repo, Power BI, Docker, Jenkins, Octopus, Git, CI/CD, NoSQL
Job PostingView all
2d
Save
Mark Applied
Hide
Data Engineer
Dallas or Alaska or California or Colorado or Delaware or Illinois or Iowa or Maryland or Massachusetts or Montana or New Hampshire or New Jersey or New York or North Dakota or Oregon or Pennsylvania or South Dakota or Vermont or West Virginia or Washington or Wyoming or Texas or Georgia or Ohio or United States or Puerto Rico
$114k-$127k/yr RemoteFull Time
Southwest Airlines
Southwest AirlinesNYSE: LUV: Operates a major commercial airline for passengers and cargo.
Design and build scalable data pipelines and analytics using cloud and SQL technologies; intermediate data engineering experience and bachelor's degree in a relevant field required.
Python, PySpark, SQL, AWS, AWS Lambda, AWS Glue, EMR, Confluent Kafka, AWS Step Functions, EventBridge, Parquet, ServiceBus, CI/CD, Unit Testing
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer (P1692)
Cincinnati or Chicago
$98k-$169k/yr OnsiteFull Time
84.51°
84.51°NYSE: KR: Provides retail data analytics and targeted advertising services.
3+ YOE3+ years data development experience with Databricks, PySpark/Spark, Python or Java, SQL/JSON, ETL and data warehousing knowledge; experience with CI/CD, pipeline orchestration, version control, and Agile.
Python, Databricks, Spark, JSON, SQL, PySpark, Java, Azure, Databricks Delta Tables, Delta Lake, Delta Live Tables, FastAPI, Flask, AKS, Kubernetes, Helm, Kafka, Azure Event Hub, Github SaaS, Github Actions, Terraform, Tableau, Power BI, Cognos
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Staff Data Engineer
Hartford or Chicago or Columbus or Charlotte
$135k-$203k/yr HybridFull Time
The Hartford
The HartfordNYSE: HIG: Provides business and personal insurance, group benefits, and investments.
8+ YOE8+ years data engineering experience; hands-on with Snowflake, Oracle, Informatica, GitHub, Qlik, SharePlex, DMS; AWS/Azure, EMR, S3, Lambda; Python/PySpark, SQL/NoSQL, CI/CD, ETL; authorized to work in the US without sponsorship.
AWS, Azure, Amazon EMR, Amazon S3, AWS Lambda, Hadoop, Apache Spark, Apache Kafka, Snowflake, Oracle, Informatica Cloud, Informatica PowerCenter, GitHub, Qlik, SharePlex, OpenFlow, DMS, PySpark, Python, SQL, NoSQL, CI/CD, Google Cloud Platform (GCP), AWS Bedrock, Vertex AI
Job PostingView all
2mo
Save
Mark Applied
Hide
Enterprise Data Engineer
Herndon or Cleveland or California or Colorado or Hawaii or Illinois or Maine or Maryland or Massachusetts or Minnesota or New Jersey or New York or Vermont or Virginia or Washington or District of Columbia
$112k-$222k/yr OnsiteFull Time
Accenture Federal Services
Accenture Federal ServicesNYSE: ACN: Provides technology and consulting services to U.S. federal agencies.
3+ YOE3+ years data engineering experience; strong Python and SQL; experience with Palantir or other ETL platforms; data integration, ETL, transformation, and data quality skills; active TS/SCI with Poly required.
Python, SQL, Palantir, AWS, Azure, GCP, Apache Spark, Hadoop, Apache Airflow, Prefect
Job PostingView all
1mo
Save
Mark Applied
Hide
Staff Data Engineer
Arizona or California or Colorado or Connecticut or District of Columbia or Delaware or Florida or Georgia or Illinois or Indiana or Kentucky or Louisiana or Massachusetts or Maine or Maryland or Michigan or Minnesota or Missouri or North Carolina or New Jersey or New York or Oregon or Pennsylvania or South Carolina or Tennessee or Texas or Virginia or Wisconsin
$130k-$150k/yr RemoteFull Time
MissionWired
MissionWired: Providing digital fundraising and data-driven marketing for nonprofit organizations.
5+ YOE5+ years professional software engineering experience; hands-on Apache Spark and distributed data pipelines; experience managing data warehouses or lakes; mentoring and guiding technical decisions; strong data modeling and production skills.
Apache Spark, AWS, GitHub Actions, Terraform, Docker
Job PostingView all
3w
Save
Mark Applied
Hide
Junior Architect / Senior Data Engineer - Remote
Schaumburg or Minneapolis or Washington or Springfield or United States
$92k-$164k/yr RemoteFull Time
UnitedHealth Group
UnitedHealth GroupNYSE: UNH: Provides health insurance and technology-enabled health care services.
5+ YOEHands-on experience with Azure Data Factory, Databricks/PySpark, Synapse/Fabric, T-SQL and Python; strong data warehouse, ETL, CI/CD, monitoring and migration skills; willing to travel ~10%.
Azure Data Factory, Azure Synapse Analytics, Databricks, Microsoft Fabric, Azure Data Lake Storage Gen2, Delta Lake, Parquet, PySpark, T-SQL, Spark SQL, KQL, PowerShell, Bash, Azure Key Vault, Azure AD, Entra ID, Azure Monitor, Log Analytics, Azure DevOps, ARM, Bicep, Terraform, Synapse Dedicated SQL Pool, Azure SQL, Teradata, Snowflake, Informatica PowerCenter, Unity Catalog, OneLake, pandas, pytest
Job PostingView all
1w
Save
Mark Applied
Hide
Data Engineer - TRANSCOM
Scott Air Force Base, Illinois, United States
$130k-$155k/yr OnsiteFull Time
Agile Defense
Agile Defense: Provides IT modernization and cybersecurity for federal government agencies.
3+ YOETop Secret clearance required, 3+ years data/software engineering experience, experience with Python, SQL, Spark/Databricks, Palantir Foundry, ML model development and deployment; bachelor’s preferred.
Palantir's MSS Workshop, Slate, Palantir Foundry, Python, SQL, Spark, Databricks, PySpark, scikit-learn, TensorFlow, XGBoost, AWS, Azure
Job PostingView all
4w
Save
Mark Applied
Hide
Senior Data Engineer
Chicago, Illinois, United States
$89k-$165k/yr HybridFull Time
Publicis Groupe
Publicis GroupeEuronext Paris: PUB: Global communications, advertising, and digital transformation holding.
5+ YOE5–8 years data engineering experience with ETL/ELT design, Apache Spark/Databricks, SQL, Python, Airflow, CI/CD and Git, experience with PostgreSQL/SparkSQL/Hive/AWS Glue, and ability to mentor engineers.
Apache Spark, Databricks, SQL, Python, Git, Airflow, PostgreSQL, SparkSQL, Hive, AWS Glue, Terraform
Job PostingView all

Explore Jobs

Expand Your Job Search