174 pyspark data engineer jobs at 108 companies in New York
1w
Save
Mark Applied
Hide
Sr. Associate, Date Engineer - PySpark
Atlanta or New York City or Philadelphia or Dallas
OnsiteFull Time
5+ YOE5+ Mgmt5+ years leading data science/engineering teams, Master’s degree in CS/DS/Statistics/Engineering, hands-on Python, PySpark, SQL, Databricks, Snowflake, experience with cloud platforms, ability to travel, U.S. work authorization required.
Python, PySpark, SQL, Databricks, Snowflake, Linux, AWS, GCP, Azure, R
2d
Save
Mark Applied
Hide
Data Engineer
New York City, New York, United States
$100k-$115k/yr HybridFull Time
2+ YOEBachelor's or master's degree in computer science or related field, 2+ years of software engineering experience, and proficiency in Python, SQL, PySpark, cloud services, data pipelines, and data warehouses.
Vertex AI, AWS Lambda, Amazon DynamoDB, Google Cloud Functions, Amazon S3, Kubernetes, AWS Glue, Google BigQuery, Amazon Web Services (AWS), Google Cloud Platform (GCP), Python, SQL, PySpark, Apache Airflow, Google Cloud Dataflow, Snowflake, Amazon Redshift, Git, JIRA, Linux, Shell, SSH, crontab, CloudWatch, Datadog, Splunk, REST, GraphQL, Docker
1mo
Save
Mark Applied
Hide
Data Engineer
New York or Los Angeles or San Francisco
$140k-$175k/yr HybridFull Time
3+ YOEBS or equivalent, 3+ years data engineering experience, Python and SQL proficiency, experience with PySpark and distributed data processing, AWS experience preferred, LLM-assisted development familiarity, and willingness to participate in on-call support.
S3, Apache Iceberg, EMR, PySpark, Dagster, Kubernetes, ClickHouse, PostgreSQL, FastAPI, Metabase, Python, SQL, AWS, AWS Athena, EHR, OMOP CDM, HIPAA, LLM
1w
Save
Mark Applied
Hide
Lead Data Engineer
New York City or United States
HybridFull Time
4+ YOERequires 4+ years in data engineering, a bachelor's or master's degree, SQL, Python, PySpark, cloud data platforms, ETL/ELT pipelines, data modeling, Airflow, cloud ecosystems, and engineering team leadership.
SQL, Python, PySpark, Snowflake, Databricks, Apache Airflow, AWS, Azure, GCP, Tableau, Microsoft Power BI, Looker, Spark, Hadoop, Hive, HBase, Kafka
3w
Save
Mark Applied
Hide
Staff Data Engineer
San Francisco or New York City
$185k-$195k/yr HybridFull Time
5+ YOE5+ years data engineering experience, strong dbt and Databricks skills, Python/PySpark, streaming and CDC ingestion, data quality and observability, experience with operational integrations.
dbt, Databricks, Python, PySpark, Delta Lake, Delta Live Tables, Airflow, Cloud Run, Cloud Functions, BigQuery, GCP, AWS, Salesforce, NetSuite, Stripe, Mulesoft, Great Expectations, Unity Catalog, Terraform, Spacelift, Boomi
1w
Save
Mark Applied
Hide
Data Engineer
New York City, New York, United States
$100k-$115k/yr HybridFull Time
2+ YOEMS or BS in computer science or related field,2+ years software engineering experience,proficiency in Python,SQL,PySpark,experience with AWS/GCP and data pipeline tools,strong problem-solving and collaboration skills.
Vertex AI, Lambdas, DynamoDB, Cloud Functions, S3, Kubernetes, Glue, BigQuery, Python, SQL, PySpark, Airflow, Dataflow, Snowflake, Redshift, IDEs, Git, JIRA, CLI, Linux, Docker
1mo
Save
Mark Applied
Hide
Data Engineer
Albany or Rome or New York
$97k-$126k/yr OnsiteFull Time
5+ YOEDesign and maintain enterprise data pipelines, warehouses/lakehouses, ETL/ELT processes using Microsoft data services, SQL, Python/PySpark; implement data governance and optimize platform performance.
Microsoft Fabric, Microsoft Azure Data Factory, Azure Data Lake Storage, Azure SQL, SQL Server Integration Services (SSIS), SQL Server, SQL, Python, PySpark
1mo
Save
Mark Applied
Hide
Senior Data Engineer
United States or San Francisco or New York City or Washington or Austin
$118k-$179k/yr RemoteFull Time
8+ YOEBachelor's in CS or equivalent, 8+ years as a software/data engineer, 5+ years building production data pipelines and Spark/PySpark experience, strong Python and SQL, cloud data warehouse and ETL tooling experience.
Spark, PySpark, Databricks, Python, SQL, Snowflake, Google BigQuery, Apache Iceberg, Fivetran, dbt, NetSuite, Salesforce, MySQL, AWS RDS/Aurora, PostgreSQL, Oracle, Microsoft SQL Server, AWS, Azure, GCP, Splunk, Datadog, AWS CloudWatch, API Gateway, Lambda, S3, SNS, SQS, SecretsManager, MCP (Model Context Protocol)
2mo
Save
Mark Applied
Hide
Senior Data Engineer
Rye, New York, United States
$127k-$137k/yr OnsiteFull Time
6+ YOE6+ years data engineering experience; bachelor’s in CS/Data Science/IT or related; expert SQL and Python (PySpark); deep Azure data platform experience (ADF, Databricks, Synapse, ADLS); data modeling, pipeline ownership, governance, and HIPAA compliance.
SQL, Python, PySpark, Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage, Microsoft Purview, Microsoft SQL Server, Oracle, CI/CD
1w
Save
Mark Applied
Hide
Senior Data Engineer
Florida or Montana or Michigan or Massachusetts or Rhode Island or West Virginia or Oregon or Oklahoma or Minnesota or New York or South Carolina or Kentucky or Virginia or Colorado or New Jersey or Wisconsin or Utah or Pennsylvania or Ohio or North Carolina or Nebraska or Mississippi or South Dakota or Maryland or Louisiana or Kansas or Iowa or Illinois or Tennessee or Delaware or Connecticut or California or Hawaii or Arkansas or District of Columbia or Georgia or Missouri or Arizona or Alabama or New Hampshire or Texas or Vermont or Indiana or Washington or Nevada or Idaho or New Mexico or Maine
$143k-$160k/yr RemoteFull Time
7+ YOE7+ years data engineering experience with 3+ years Databricks; strong Spark/PySpark, Delta Lake, SQL, Azure, Git and CI/CD skills; Bachelor's degree required.
Databricks, Apache Spark, Delta Lake, Unity Catalog, Workflows, SQL Warehouses, Delta Live Tables, PySpark, SQL, Azure, Git, dbt, Terraform, Azure DevOps
2w
Save
Mark Applied
Hide
Data Engineer
New York, New York, United States
$72k-$97k/yr OnsiteFull Time
Design Lakehouse architectures on Databricks, develop PySpark jobs, implement governance with Unity Catalog, work with Airflow and CI/CD; strong data governance and streaming knowledge.
Databricks, Pyspark, CI/CD, Airflow, Unity Catalog, Databricks Apps, Delta Sharing
2w
Save
Mark Applied
Hide
Databricks Data Engineer
United States or Arlington or Tysons or Washington or New York City or Chicago or Austin or Atlanta or Boston or Boulder
$113k-$188k/yr RemoteFull Time
3+ YOEBachelor's degree,3+ years data engineering experience,proficiency with Python/PySpark/SQL,Databricks/Spark/Delta Lake experience,knowledge of data pipelines,governance,and cloud platforms.
Databricks, PySpark, Python, SQL, Delta Lake, Unity Catalog, Spark, CI/CD, Terraform, Azure, AWS, GCP
2w
Save
Mark Applied
Hide
Data Analytics Engineer
Farmingdale, New York, United States
HybridFull Time
3+ YOE3+ years data engineering experience; strong Python, PySpark, and SQL; building ELT/ETL pipelines, analytics data modeling, Microsoft Fabric/Power BI familiarity; Bachelor's in CS/Data or equivalent.
Python, PySpark, SQL, Microsoft Fabric, Microsoft Power BI, Azure Synapse, Azure Data Lake, dbt
3w
Save
Mark Applied
Hide
Senior Data Engineer
New York City or San Francisco
$170k-$200k/yr HybridFull Time
5+ YOE5+ years building and operating production ETL pipelines; strong Python and SQL skills; Spark/PySpark, Databricks/Delta Lake, Airflow experience; strong data modeling and data quality practices.
Python, SQL, Spark, PySpark, Databricks, Delta Lake, Airflow, ClickHouse
2mo
Save
Mark Applied
Hide
Data Engineer
Jersey City or Irving or New York City or Tampa
$142k-$213k/yr HybridFull Time
5+ YOE5-8 years experience, Bachelor’s degree or equivalent, strong experience in data warehouse/ETL, AWS Glue/Athena, Apache Airflow, Snowflake, Python/PySpark, Unix shell scripting, Ab Initio, and database systems; mortgage domain experience preferred.
AWS Glue, AWS Athena, Apache Airflow, Snowflake, MongoDB, Oracle, PostgreSQL, Unix shell scripting, Python, PySpark, Ab Initio (EME, GDE, Co-op)
1mo
Save
Mark Applied
Hide
Senior Data Engineer
New York, New York, United States
$140k-$160k/yr HybridFull Time
5+ YOE5+ years building production Spark-based data pipelines in cloud environments; expertise in PySpark and SQL; experience with Databricks, orchestration (Airflow/Dagster), CI/CD, EMR/Dataproc, data governance, and collaboration using Git.
Delta Lake, Databricks Auto Loader, Structured Streaming, Unity Catalog, Photon, Databricks Workflows, Delta Live Tables, Airflow, Dagster, MLflow, Presto, PySpark, Spark, Databricks, EMR, Dataproc, Git, ChatGPT, Lakeflow Spark Declarative Pipelines (SDP)
2mo
Save
Mark Applied
Hide
Sr. Engineer, Data - Archimedes
St. Louis or Brentwood or United States or Alaska or Arizona or Arkansas or California or Connecticut or Delaware or Hawaii or Idaho or Louisiana or Maine or Massachusetts or Michigan or Mississippi or Montana or Nebraska or Nevada or New Mexico or New York or North Carolina or North Dakota or Oregon or Rhode Island or South Carolina or South Dakota or Texas or Utah or Vermont or Washington or West Virginia or Wyoming
HybridFull Time
8+ YOEBachelor's in CS or related (Master's preferred); AWS/Azure/CDMP certifications; 8+ years data engineering experience, 5+ years building lakehouse solutions on Azure Databricks; expertise in Spark, PySpark, SQL, Python, DataOps, CI/CD, and cloud data services.
Azure Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage Gen2, Databricks Workflows, Databricks Asset Bundles, Delta Live Tables (DLT), Synapse Analytics, Azure Data Factory (ADF), Azure Functions, Azure Key Vault, Azure Monitor, Managed Identities, Event Hub, Event Grid, Service Bus, Apache Spark, PySpark, SQL, Python, Azure DevOps, GitHub Actions, Jenkins, AWS Glue, AWS Lambda, Amazon S3, Amazon RDS, Amazon Redshift, Apache Airflow, Git, Terraform, PowerShell, MLflow, Azure Machine Learning, Databricks ML, REST APIs, GraphQL, CloudWatch, ELK Stack
1mo
Save
Mark Applied
Hide
Data Engineer (f/m/x)
Munich or New York City or Denver or Chicago or London or Heidelberg or Singapore
OnsiteFull Time
2+ YOE2+ years data engineering experience; strong Python, Pyspark and SQL skills; ETL/ELT and data pipeline architecture; exposure to cloud (AWS/Azure/GCP), Git and CI/CD; good communication and problem-solving.
Python, Pyspark, SQL, Databricks, Snowflake, SQL Server, AWS, Redshift, S3, Glue, Lambda, Azure DevOps, Azure Repo, Power BI, Docker, Jenkins, Octopus, Git, CI/CD, NoSQL
1w
Save
Mark Applied
Hide
Senior Data Engineer, AI Systems
New York City or Toronto
$165k-$215k/yr OnsiteFull Time
5+ YOE5+ years data engineering experience with Apache Spark/PySpark, GCP Dataproc, Python, Parquet/Delta Lake, Kafka, and cloud/CI/CD tooling; strong optimization and collaboration skills.
Apache Spark, PySpark, GCP Dataproc, Parquet, Delta Lake, Kafka, Google Cloud Platform, Python, Docker, Kubernetes, GitHub Actions, Git
2mo
Save
Mark Applied
Hide
Data Engineer
Jersey City or Irving or New York or Tampa
$142k-$213k/yr HybridFull Time
5+ YOE5-8 years of data engineering experience; strong ETL, data warehousing, Python, Spark; AWS Glue, Snowflake; SQL databases; project experience.
AWS Glue, AWS Athena, Airflow, Snowflake, MongoDB, Oracle, PostgreSQL, Unix Shell, Python, PySpark, Ab Initio
Explore Jobs
Expand Your Job Search
Top Cities in New York for Pyspark Data Engineer
- Pyspark Data Engineer jobs in New York, NY
- Pyspark Data Engineer jobs in Brooklyn, NY
- Pyspark Data Engineer jobs in Queens, NY
- Pyspark Data Engineer jobs in Manhattan, NY
- Pyspark Data Engineer jobs in The Bronx, NY
- Pyspark Data Engineer jobs in Staten Island, NY
- Pyspark Data Engineer jobs in Buffalo, NY
- Pyspark Data Engineer jobs in Rochester, NY
- Pyspark Data Engineer jobs in Yonkers, NY
- Pyspark Data Engineer jobs in Syracuse, NY
- Pyspark Data Engineer jobs in Amherst, NY
- Pyspark Data Engineer jobs in Albany, NY
- Pyspark Data Engineer jobs in Greenburgh, NY
- Pyspark Data Engineer jobs in New Rochelle, NY
- Pyspark Data Engineer jobs in Cheektowaga, NY
- Pyspark Data Engineer jobs in Mount Vernon, NY
- Pyspark Data Engineer jobs in Schenectady, NY
- Pyspark Data Engineer jobs in Utica, NY
- Pyspark Data Engineer jobs in White Plains, NY
- Pyspark Data Engineer jobs in Clay, NY