181 pyspark data engineer jobs at 103 companies in Babylon, NY
1w
Save
Mark Applied
Hide
Big Data PySpark Lead Engineer - Vice President
Jersey City, New Jersey, United States
$142k-$213k/yr HybridFull Time
6+ YOEExpertise in PySpark and Hadoop ecosystem, strong SQL, data modeling, shell/Autosys automation, 6+ years apps development/systems analysis experience, bachelor’s degree or equivalent.
PySpark, Hive, HDFS, Sqoop, Spark, Impala, Scala, SQL, Autosys, Apache Kafka, Shell scripting
2w
Save
Mark Applied
Hide
Hadoop PySpark Data Engineer
Charlotte or Jersey City or Newark or Plano
$82k-$127k/yr OnsiteFull Time
Experience with Hadoop ecosystem (Cloudera), PySpark, Spark, Hive, Python, Unix shell scripting, Autosys/Airflow, SQL; Bachelor\u0002s or equivalent experience; ability to work across SDLC and travel/relocate as needed.
Hadoop, Cloudera, PySpark, Spark, Hive, Python, Unix shell scripting, Autosys, Airflow, SQL, Tableau
1w
Save
Mark Applied
Hide
Big Data PySpark Lead Engineer - Vice President
Jersey City, New Jersey, United States
$142k-$213k/yr HybridFull Time
6+ YOEExpertise in PySpark and Hadoop ecosystem, complex SQL, data modeling, Autosys and shell scripting; 6+ years apps development or systems analysis experience; bachelor’s degree preferred.
PySpark, Hadoop, Hive, HDFS, Sqoop, Spark, Impala, Scala, SQL, Autosys, Apache Kafka, Shell scripting
3w
Save
Mark Applied
Hide
Data Engineer III - (Pyspark/databricks/Mongo/Sql/Python)
Jersey City, New Jersey, United States
$119k-$160k/yr OnsiteFull Time
5+ YOE5+ years data engineering experience across the data lifecycle, advanced SQL, NoSQL (Mongo) familiarity, PySpark/Databricks and AWS exposure preferred; strong analytical and tool-customization skills.
Pyspark, Databricks, Mongo, SQL, AWS, ECS, Lambdas, EKS, Java
1w
Save
Mark Applied
Hide
Sr. Associate, Date Engineer - PySpark
Atlanta or New York City or Philadelphia or Dallas
OnsiteFull Time
5+ YOE5+ Mgmt5+ years leading data science/engineering teams, Master’s degree in CS/DS/Statistics/Engineering, hands-on Python, PySpark, SQL, Databricks, Snowflake, experience with cloud platforms, ability to travel, U.S. work authorization required.
Python, PySpark, SQL, Databricks, Snowflake, Linux, AWS, GCP, Azure, R
1mo
Save
Mark Applied
Hide
Data Engineer
New York or Los Angeles or San Francisco
$140k-$175k/yr HybridFull Time
3+ YOEBS or equivalent, 3+ years data engineering experience, Python and SQL proficiency, experience with PySpark and distributed data processing, AWS experience preferred, LLM-assisted development familiarity, and willingness to participate in on-call support.
S3, Apache Iceberg, EMR, PySpark, Dagster, Kubernetes, ClickHouse, PostgreSQL, FastAPI, Metabase, Python, SQL, AWS, AWS Athena, EHR, OMOP CDM, HIPAA, LLM
6d
Save
Mark Applied
Hide
Lead Data Engineer
New York City or United States
HybridFull Time
4+ YOERequires 4+ years in data engineering, a bachelor's or master's degree, SQL, Python, PySpark, cloud data platforms, ETL/ELT pipelines, data modeling, Airflow, cloud ecosystems, and engineering team leadership.
SQL, Python, PySpark, Snowflake, Databricks, Apache Airflow, AWS, Azure, GCP, Tableau, Microsoft Power BI, Looker, Spark, Hadoop, Hive, HBase, Kafka
3w
Save
Mark Applied
Hide
Staff Data Engineer
San Francisco or New York City
$185k-$195k/yr HybridFull Time
5+ YOE5+ years data engineering experience, strong dbt and Databricks skills, Python/PySpark, streaming and CDC ingestion, data quality and observability, experience with operational integrations.
dbt, Databricks, Python, PySpark, Delta Lake, Delta Live Tables, Airflow, Cloud Run, Cloud Functions, BigQuery, GCP, AWS, Salesforce, NetSuite, Stripe, Mulesoft, Great Expectations, Unity Catalog, Terraform, Spacelift, Boomi
1mo
Save
Mark Applied
Hide
Data Engineer
Albany or Rome or New York
$97k-$126k/yr OnsiteFull Time
5+ YOEDesign and maintain enterprise data pipelines, warehouses/lakehouses, ETL/ELT processes using Microsoft data services, SQL, Python/PySpark; implement data governance and optimize platform performance.
Microsoft Fabric, Microsoft Azure Data Factory, Azure Data Lake Storage, Azure SQL, SQL Server Integration Services (SSIS), SQL Server, SQL, Python, PySpark
1d
Save
Mark Applied
Hide
Sr Data Engineer
New York City, New York, United States
$135k-$194k/yr RemoteFull Time
5+ YOEBachelor's degree or equivalent experience; 8+ years IT experience, 5+ years enterprise data engineering, Python, PySpark, AWS, SQL, data pipelines, warehousing, cloud platforms, and Agile methods.
Amazon Web Services (AWS), Python, PySpark, Amazon EMR, AWS Glue, Apache Spark, dbt, Microsoft Azure, Google Cloud Platform, SQL, Git, Apache Iceberg, Amazon S3, Amazon Athena, Amazon Redshift, AWS Lambda, AWS Lake Formation, AWS Identity and Access Management (IAM), Amazon CloudWatch, Terraform, Apache Airflow, Managed Workflows for Apache Airflow (MWAA), AWS Step Functions, Kafka, Amazon Kinesis, Spark Structured Streaming, RESTful APIs, Hadoop, NoSQL, CI/CD, DataOps, ELT, ETL, APIs
1mo
Save
Mark Applied
Hide
Senior Data Engineer
United States or San Francisco or New York City or Washington or Austin
$118k-$179k/yr RemoteFull Time
8+ YOEBachelor's in CS or equivalent, 8+ years as a software/data engineer, 5+ years building production data pipelines and Spark/PySpark experience, strong Python and SQL, cloud data warehouse and ETL tooling experience.
Spark, PySpark, Databricks, Python, SQL, Snowflake, Google BigQuery, Apache Iceberg, Fivetran, dbt, NetSuite, Salesforce, MySQL, AWS RDS/Aurora, PostgreSQL, Oracle, Microsoft SQL Server, AWS, Azure, GCP, Splunk, Datadog, AWS CloudWatch, API Gateway, Lambda, S3, SNS, SQS, SecretsManager, MCP (Model Context Protocol)
2mo
Save
Mark Applied
Hide
Senior Data Engineer
Rye, New York, United States
$127k-$137k/yr OnsiteFull Time
6+ YOE6+ years data engineering experience; bachelor’s in CS/Data Science/IT or related; expert SQL and Python (PySpark); deep Azure data platform experience (ADF, Databricks, Synapse, ADLS); data modeling, pipeline ownership, governance, and HIPAA compliance.
SQL, Python, PySpark, Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage, Microsoft Purview, Microsoft SQL Server, Oracle, CI/CD
2w
Save
Mark Applied
Hide
Data Engineer
New York, New York, United States
$72k-$97k/yr OnsiteFull Time
Design Lakehouse architectures on Databricks, develop PySpark jobs, implement governance with Unity Catalog, work with Airflow and CI/CD; strong data governance and streaming knowledge.
Databricks, Pyspark, CI/CD, Airflow, Unity Catalog, Databricks Apps, Delta Sharing
1mo
Save
Mark Applied
Hide
Senior Data Engineer
United States or San Francisco or New York City
$120k-$235k/yr OnsiteFull Time
3+ YOEMaster's (3+ yrs) or Bachelor's (4+ yrs) in CS/Math/Engineering or equivalent experience; experience building production data systems, instrumentation/telemetry, experimentation, PySpark/SQL, and mentoring engineers; must pass Microsoft background checks.
ADLS, Azure Synapse, Microsoft Fabric, Databricks, PySpark, SQL, Cosmos, Scope, Conform, Aria
2w
Save
Mark Applied
Hide
Data Analytics Engineer
Farmingdale, New York, United States
HybridFull Time
3+ YOE3+ years data engineering experience; strong Python, PySpark, and SQL; building ELT/ETL pipelines, analytics data modeling, Microsoft Fabric/Power BI familiarity; Bachelor's in CS/Data or equivalent.
Python, PySpark, SQL, Microsoft Fabric, Microsoft Power BI, Azure Synapse, Azure Data Lake, dbt
1mo
Save
Mark Applied
Hide
Sr Data Engineer
Shelton, Connecticut, United States
$103k-$128k/yr OnsiteFull Time
3+ YOEBachelor's required; 3+ years data engineering experience; hands-on Databricks or Snowflake lakehouse expertise; PySpark, Python, SQL; streaming (DLT/Snowpipe), Airflow, dbt, Git, cloud (AWS/Azure/GCP); strong communication.
Databricks, Snowflake, Delta Lake, Iceberg, Medallion, PySpark, SQL, Python, Databricks Structured Streaming, Databricks DLT, Snowflake Dynamic Tables, Snowpipe, Databricks Mosaic AI, Snowflake Cortex, Airflow, Databricks Lakeflow, Snowflake Openflow, dbt, Unity Catalog, Horizon Catalog, Databricks AI/BI Genie, Unity Catalog Metrics, Snowflake Semantic Views, Cortex Analyst, Git, Databricks Asset Bundles, Snowflake CLI, Schemachange, AWS, S3, Glue, Kinesis, Azure, GCP
2w
Save
Mark Applied
Hide
Sr. Engineer, Data & Analytics
Stamford or United States
RemoteFull Time
5+ YOEBachelor's in CS/Data/Engineering,5+ years in data engineering and analytics,expertise in Microsoft Power BI and Microsoft Fabric,SQL,Python/PySpark,DAX,Data modeling,CI/CD and stakeholder partnership.
Microsoft Power BI, Microsoft Fabric, OneLake, Microsoft Data Factory, Dataflows Gen2, Spark, SQL, Python, PySpark, Direct Lake, Git, Microsoft Purview, Databricks, Snowflake, Redshift, BigQuery, DAX, Delta Lake, Parquet
2w
Save
Mark Applied
Hide
Senior Data Engineer
New York City or San Francisco
$170k-$200k/yr HybridFull Time
5+ YOE5+ years building and operating production ETL pipelines; strong Python and SQL skills; Spark/PySpark, Databricks/Delta Lake, Airflow experience; strong data modeling and data quality practices.
Python, SQL, Spark, PySpark, Databricks, Delta Lake, Airflow, ClickHouse
1mo
Save
Mark Applied
Hide
Senior Data Engineer
New York, New York, United States
$140k-$160k/yr HybridFull Time
5+ YOE5+ years building production Spark-based data pipelines in cloud environments; expertise in PySpark and SQL; experience with Databricks, orchestration (Airflow/Dagster), CI/CD, EMR/Dataproc, data governance, and collaboration using Git.
Delta Lake, Databricks Auto Loader, Structured Streaming, Unity Catalog, Photon, Databricks Workflows, Delta Live Tables, Airflow, Dagster, MLflow, Presto, PySpark, Spark, Databricks, EMR, Dataproc, Git, ChatGPT, Lakeflow Spark Declarative Pipelines (SDP)
1mo
Save
Mark Applied
Hide
Data Engineer (f/m/x)
Munich or New York City or Denver or Chicago or London or Heidelberg or Singapore
OnsiteFull Time
2+ YOE2+ years data engineering experience; strong Python, Pyspark and SQL skills; ETL/ELT and data pipeline architecture; exposure to cloud (AWS/Azure/GCP), Git and CI/CD; good communication and problem-solving.
Python, Pyspark, SQL, Databricks, Snowflake, SQL Server, AWS, Redshift, S3, Glue, Lambda, Azure DevOps, Azure Repo, Power BI, Docker, Jenkins, Octopus, Git, CI/CD, NoSQL
Explore Jobs