HiringCafe
Join our community

70 spark engineer jobs at 37 companies in Philadelphia, PA

1mo
Save
Mark Applied
Hide
Principal Data Engineer
Philadelphia, Pennsylvania, United States
HybridFull Time
Medical Guardian
Medical Guardian: Private digital health providing medical alert, remote monitoring, and connected-care services for aging adults.
10+ YOE3+ Mgmt10+ years in data or software engineering, 7+ years building production data pipelines, strong Databricks/Spark/PySpark/SQL experience, Azure and streaming expertise, plus leadership and production operations experience.
Databricks, PySpark, Spark SQL, Spark, Python, SQL, Delta Lake, Databricks Workflows, CI/CD, Azure, Azure Event Hubs, Azure Stream Analytics, Azure Service Bus, Azure Data Factory, Azure Functions, ADLS, Azure Cosmos DB, Event Grid
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Bala Cynwyd, Pennsylvania, United States
OnsiteContract
Susquehanna International Group
Susquehanna International Group: Global quantitative trading and technology firm.
5+ YOE5+ years in data engineering with strong Python and experience with modern data platforms (Spark, Databricks, Trino, Iceberg, Snowflake, Kafka); capital markets/financial services experience required.
Spark, Databricks, Trino, Iceberg, Snowflake, Parquet, Apache Arrow, Polars, Kafka, Python
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Bala Cynwyd, Pennsylvania, United States
OnsiteContract
Susquehanna International Group
Susquehanna International Group: Global quantitative trading and technology firm.
5+ YOE5+ years data engineering experience in large-scale distributed processing; strong Python; experience with Spark/Databricks, Trino, Iceberg, Snowflake, Parquet, Apache Arrow, Polars, and/or Kafka; capital markets domain experience.
Spark, Databricks, Trino, Iceberg, Snowflake, Parquet, Apache Arrow, Polars, Kafka, Python
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer, Clinical Operations
Princeton, New Jersey, United States
$88k-$106k/yr OnsiteFull Time
Bristol Myers Squibb
Bristol Myers SquibbNYSE: BMY: Transforming patients' lives through science.
5+ YOE5+ years data engineering experience with Databricks, cloud-native data platforms, Python/SQL/Spark, GenAI/LLM experience, data pipeline and governance expertise for clinical/cross-study data.
Generative AI, Databricks, Databricks Delta Lake, Databricks Unity Catalog, Databricks Workflows, Databricks Mosaic AI, MLflow, Python, SQL, Spark, PySpark, RAG, vector embeddings
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Bala Cynwyd, Pennsylvania, United States
OnsiteFull Time
Susquehanna International Group
Susquehanna International Group: Global quantitative trading and technology firm.
5+ YOE5+ years data engineering experience, capital markets domain knowledge, strong Python skills, experience with Spark/Databricks/Trino/Snowflake/Kafka and large-scale batch/stream processing, ability to support off-hours/weekend needs.
Spark, Databricks, Trino, Iceberg, Snowflake, Parquet, Apache Arrow, Polars, Kafka, Python
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer, Clinical Operations
Princeton, New Jersey, United States
$88k-$106k/yr OnsiteFull Time
Bristol Myers Squibb
Bristol Myers SquibbNYSE: BMY: Transforming patients' lives through science.
5+ YOE5+ years data engineering experience with cloud platforms, Databricks (Delta Lake, Unity Catalog), Python, SQL, Spark/PySpark, GenAI/LLM (RAG, embeddings), and building production-grade ETL/ELT pipelines and data products.
Generative AI (GenAI), Databricks, Databricks Delta Lake, Databricks Unity Catalog, Databricks Workflows, Databricks Mosaic AI, MLflow, Python, SQL, Spark, PySpark, RAG, LLM
Job PostingView all
4w
Save
Mark Applied
Hide
Security Data Solutions Engineer
Quincy or Princeton or Irvine or Boston or Austin or Sacramento
$120k-$203k/yr HybridFull Time
State Street
State StreetNYSE: STT: Global financial services and bank holding.
8+ YOEDesign and implement enterprise security telemetry and data products; 8+ years in cybersecurity/data engineering; strong Python, SQL, Spark, Databricks, Snowflake, SIEM experience; product management skills.
Splunk, Microsoft Sentinel, QRadar, Elastic, Chronicle, Databricks, Snowflake, Kafka, Spark, Cribl, OpenTelemetry, FluentBit, Python, SQL, AWS, Azure, Google Cloud
Job PostingView all
3w
Save
Mark Applied
Hide
Distinguished Engineer -Developer Insights & AI Telemetry
New York City or McLean or Wilmington or Richmond
$245k-$335k/yr OnsiteFull Time
Capital One
Capital OneNYSE: COF: A technology-driven bank providing diverse financial services.
7+ YOEBachelor’s degree and 7+ years in software, solution, or enterprise architecture; 5+ years in big data; 4+ years in production ML/AI; 2+ years in cloud computing.
Spark, Kafka, Kinesis, ClickHouse, Snowflake, AWS, Microsoft Azure, Google Cloud, Python, Go, Java, TypeScript, OpenTelemetry, Protobuf, DORA, SPACE
Job PostingView all
3w
Save
Mark Applied
Hide
Distinguished Engineer -Developer Insights & AI Telemetry
New York City or McLean or Richmond or Wilmington
$294k-$335k/yr OnsiteFull Time
Capital One
Capital OneNYSE: COF: A technology-driven bank providing diverse financial services.
7+ YOEBachelor’s degree and 7+ years in software, solution, or enterprise architecture; expertise in big data, streaming, ML/AI, cloud computing, distributed systems, and technical leadership.
Spark, Kafka, Kinesis, ClickHouse, Snowflake, AWS, Microsoft Azure, Google Cloud, Python, Go, Java, TypeScript, OpenTelemetry, Protobuf, DORA, SPACE
Job PostingView all
1mo
Save
Mark Applied
Hide
Lead Software Engineer - Python, Databricks and AWS
Jersey City or Wilmington
$152k-$215k/yr OnsiteFull Time
JPMorgan Chase
JPMorgan ChaseNYSE: JPM: Global financial services and investment banking firm.
5+ YOE5+ years SDLC experience, strong data engineering with Databricks Lakehouse, Delta Lake, Spark; AWS (S3, IAM, KMS); streaming and batch pipelines; CI/CD and Terraform; governance and monitoring.
Databricks, Delta Lake, Spark, Delta Live Tables (DLT), Databricks Workflows, Structured Streaming, AWS, S3, IAM, KMS, Unity Catalog, Terraform
Job PostingView all
4d
Save
Mark Applied
Hide
Responsible AI Engineer
New York City or Milwaukee or Dallas or Columbus or Kirkland or Cincinnati or Cleveland or Oklahoma City or Austin or Albany or Chicago or St. Petersburg or Hartford or Pittsburgh or St. Louis or Miami or Sacramento or Raleigh or Minneapolis or Mountain View or Scottsdale or San Francisco or Morristown or Denver or Boston or Philadelphia or Des Moines or Overland Park or Los Angeles or Charlotte or Walnut Creek or Carmel or Seattle or Houston or Arlington or Atlanta or Redmond or Bentonville or Beaverton or Nashville or Detroit or San Diego
$68k-$220k/yr HybridFull Time
Accenture
AccentureNYSE: ACN: Global professional services delivering 360° value.
3+ YOEBachelor's degree or equivalent experience, 3+ years developing AI and machine learning solutions, 3+ years with programming, frameworks, APIs, version control, databases, and cloud platforms; responsible AI experience preferred.
Python, R, Java, Scikit-learn, TensorFlow, PyTorch, Django, Flask, FastAPI, GitHub, Bitbucket, AWS, Azure, Google Cloud, Hadoop, Spark
Job PostingView all
3w
Save
Mark Applied
Hide
Senior Data Engineer
Austin or Boston or Charleston or Charlotte or Chicago or Dallas or Durham or Harrisburg or Houston or Irvine or Kansas City or Los Angeles or Miami or Nashville or New York or Newark or Palo Alto or Pittsburgh or Portland or Raleigh or San Francisco or Seattle or Washington or Wilmington
$128k-$249k/yr HybridFull Time
K&L Gates
K&L Gates: Global law firm providing corporate, regulatory, litigation, and policy services to companies, public entities, institutions, and individuals.
5+ YOE5+ years designing enterprise Microsoft Fabric data platforms; strong SQL and Python skills; experience with Spark, CI/CD, Azure DevOps/GitHub Actions; knowledge of data governance and Azure AI Foundry; Bachelor's degree or equivalent.
Microsoft Fabric, OneLake, Fabric Data Factory, Dataflow Gen2, Fabric Notebooks, Azure AI Foundry, Microsoft Copilot, Claude, SQL, Python, Spark, Azure DevOps, GitHub Actions, Microsoft Purview
Job PostingView all
1mo
Save
Mark Applied
Hide
Hadoop and PySpark Data Engineer
Charlotte or Jacksonville or Jersey City or Newark or Plano
OnsiteFull Time
Infosys
InfosysNYSE: INFY: Global leader in next-generation digital services and consulting.
Experience with Hadoop, PySpark, Hive, Spark, Python, SQL, Autosys/Airflow, Unix shell; bachelor\u0002s degree or equivalent; strong communication and SDLC experience required.
Hadoop, PySpark, Hive, Spark, Python, SQL, Unix Shell, Autosys, Airflow, Cloudera, Tableau
Job PostingView all
3w
Save
Mark Applied
Hide
Data Engineer
MetroPark, New Jersey, United States
$88k-$105k/yr HybridFull Time
Mizuho Financial Group
Mizuho Financial GroupTokyo Stock Exchange: 8411: Japanese publicly traded bank holding providing banking, securities, trust, asset-management, and research services.
0+ YOERequires 0–2 years in data engineering, analytics, or related technical work; SQL, Python, OOP, data concepts, and bachelor's degree in a related field. Experience with Databricks, Spark, cloud, ETL, and databases is valued.
Databricks, Microsoft Azure, Apache Spark, PySpark, SQL, Python, Git, Delta Lake, Amazon Web Services (AWS), Google Cloud Platform (GCP), Scala, SSIS, Azure Data Factory, Kafka
Job PostingView all
2w
Save
Mark Applied
Hide
Hadoop and PySpark Data Engineer
Charlotte or Jacksonville or Jersey City or Newark or Plano or Delaware or Florida or New Jersey or North Carolina or Texas or United States
$82k-$127k/yr OnsiteFull Time
Infosys
InfosysNYSE: INFY: Global leader in next-generation digital services and consulting.
Bachelor's degree or equivalent experience; Hadoop, Hive, Spark, PySpark, Python, Unix shell scripting, SQL, data warehousing, orchestration tools, Agile, and production Big Data experience required.
Hadoop, PySpark, Cloudera, Hive, Spark, Python, Unix, Autosys, Airflow, SQL, Agile, Tableau, AI, cloud, containerization
Job PostingView all
1mo
Save
Mark Applied
Hide
Principal Machine Learning Engineer
Philadelphia, Pennsylvania, United States
OnsiteContract
Delan Associates, Inc
Delan Associates, Inc: Professional services and engineering consulting firm.
5+ YOE5+ years ML model development, 3+ years production model operationalization, strong Python and SQL skills, experience with Databricks/Spark/MLflow, model evaluation and MLOps, and stakeholder communication.
Python, SQL, Spark, Databricks, MLflow, scikit-learn, XGBoost, Snowflake, Azure, AWS
Job PostingView all
1w
Save
Mark Applied
Hide
AI Engineer
New York City or United States or Raleigh-Durham or Atlanta or Philadelphia
$150k-$300k/yr HybridFull Time
Sciemo
Sciemo: AI revenue-growth platform helping consumer brands optimize pricing, promotions, assortment, and supply-chain decisions.
Production experience launching LLM or agent products, strong Python, agent frameworks, RAG, prompt engineering, evaluation systems, ML tools, orchestration workflows, and stakeholder communication.
Python, AWS, PyTorch, scikit-learn, SQL, Spark, Airflow, ZenML, Kedro, dbt
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Cloud Ops and Dev Ops Engineer
Cambridge or Plymouth Meeting or United States
OnsiteFull Time
ConcertAI
ConcertAI: Private AI and real-world-data SaaS serving healthcare and life-sciences organizations, especially oncology.
4+ YOE4+ years operating data management solutions on cloud, experience with AWS services, CI/CD, Terraform, database technologies, HIPAA-aware data handling, and onshore/offshore collaboration.
Jenkins, GitHub Actions, Terraform, S3, EFS, EC2, RDS, EKS, Glue, Secrets Manager, AWS Batch, PostgreSQL, MongoDB, Redshift, Elasticsearch, Spark, Bash, Git, LLMOps, GenAI LLM
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Databricks Engineer
Malvern, Pennsylvania, United States
$120k-$135k/yr OnsiteFull Time
Tata Consultancy Services
Tata Consultancy ServicesBSE: 532540: Global leader in IT services, consulting, and business solutions.
10+ YOEExpertise in Apache Spark, Delta Lake, Databricks (DLT, Workflows, Unity Catalog), Python/PySpark and SQL; experience with AWS (S3, Redshift, IAM, KMS); 10+ years experience and bachelor in computer science.
Apache Spark, Spark SQL, DataFrame API, Delta Lake, Delta Live Tables (DLT), Databricks Workflows, Unity Catalog, Python, PySpark, SQL, Amazon S3, Amazon Redshift, AWS KMS, Databricks Git Folders, Git, MLflow, AWS Glue, Amazon Kinesis, AWS Step Functions
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Philadelphia, Pennsylvania, United States
$68k-$153k/yr OnsiteFull Time
Universal Music Group
Universal Music GroupEuronext Amsterdam: UMG: Public Dutch music-entertainment producing and publishing recorded music, merchandise, and audiovisual content for artists and audiences.
5+ YOE5+ years data engineering experience building scalable pipelines, ecommerce/Shopify data expertise, strong SQL and Python, cloud and modern data platform experience, semantic layer and AI/ML data readiness skills.
Shopify, dbt, LookML, Spark, Snowflake, Databricks, BigQuery, Redshift, Parquet, SQL, Python, AWS, Azure, GCP
Job PostingView all

Explore Jobs

Expand Your Job Search