301 spark data engineer jobs at 156 companies in Merrick, NY
2d
Save
Mark Applied
Hide
2d
Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI)
McLean or New York City or Richmond
$179k-$246k/yrOnsiteFull Time
Capital OneNYSE: COF: Financial services offering credit cards, banking, and loans.
4+ YOEBachelor's degree, 4+ years application development, 2+ years big data, 1+ year cloud (AWS/Azure/GCP); experience with Python, SQL, Spark, data modeling, and cloud data platforms.
Bedrock Robotics: Automates heavy construction machinery with AI retrofit kits.
5+ YOE5+ years data engineering experience in large-scale data lake/warehouse environments; strong SQL and distributed query engine experience; pipeline orchestration (Airflow/Prefect) and Spark/Databricks experience; data quality and observability expertise.
Sr Data Engineer, Python + Spark (Data Federation skillset - Data Lakehouse - Eg: Starburst) - New York
New York, New York, United States
$40k-$140k/yrOnsiteFull Time
Photon: Global technology services provider.
5+ YOESenior Data Engineer with 5+ years in Python, Spark; expertise in data federation, lakehouse architectures (Delta Lake/Iceberg/Hudi); Starburst/Trino/Dremio; cloud platforms (AWS/Azure/GCP); strong SQL and data modeling.
FanDuelNYSE: FLUT: Offers online sports betting and daily fantasy sports services.
3+ YOE3+ years in data or software engineering with strong SQL, Python/Java/Scala, Databricks, Airflow, dbt, Spark, Kafka, and cloud (AWS/GCP/Azure) experience.
United States or San Francisco or New York City or Washington or Austin
$118k-$179k/yrRemoteFull Time
SamsaraNYSE: IOT: Connected Operations Cloud platform for physical operations and IoT.
8+ YOEBachelor's in CS or equivalent, 8+ years as a software/data engineer, 5+ years building production data pipelines and Spark/PySpark experience, strong Python and SQL, cloud data warehouse and ETL tooling experience.
VersantNasdaq: VSNT: Operates cable television networks and digital media entertainment platforms.
5+ YOE5+ years building production data pipelines with Spark-based platforms, expertise in PySpark and SQL, Databricks experience preferred, bachelor's in CS/data engineering or equivalent, strong CI/CD, orchestration, and data governance skills.
Delta Lake, Databricks Auto Loader, Databricks, Structured Streaming, Change Data Capture (CDC), Unity Catalog, Spark, Photon, Databricks Workflows, Delta Live Tables, Airflow, MLflow, Presto, Flink, Git, Dagster, EMR, Dataproc, Lakeflow Spark Declarative Pipelines (SDP), ChatGPT
EXLNASDAQ: EXLS: Provides data analytics and digital operations solutions to businesses.
10+ YOE10+ years data engineering experience with Databricks, Spark, Delta Lake and Azure Data Platform; strong Python/PySpark and SQL skills; experience with CI/CD, REST APIs, CDC/SCD, and data quality frameworks.
Databricks, Spark, Delta Lake, Unity Catalog, Databricks Workflows, SQL Warehouses, PySpark, Spark SQL, SQL, Python, Azure Data Lake Storage (ADLS Gen2), Azure Synapse, Azure Key Vault, Azure DevOps, Git, Ab Initio, REST APIs, JSON, XML
CognizantNasdaq: CTSH: Provides global information technology and business process outsourcing services.
5+ YOE5+ years data engineering experience; strong Python, SQL, Spark; experience with Snowflake, DBT and AWS/Azure; exposure to real-time streaming (Kafka/Kinesis); collaborate with data science and analytics teams.
Reality Defender: Detects deepfakes and AI-generated media to identify fraud.
Hands-on experience with Kubernetes and AWS, distributed computing (Spark, Ray), workflow orchestration (Airflow), Python and SQL; experience building multi-terabyte and streaming data pipelines for audio/video is preferred.
Linero: AI-native, venture-backed building data infrastructure software.
5+ YOE5+ years data engineering; Airflow, dbt, Spark; streaming and batch processing; ontology design; multi-tenant, high-security data environments; based in or relocate to New York City.
Balyasny Asset Management: Global multi-strategy investment firm managing diverse alternative asset classes.
3+ YOE3+ years data engineering experience; strong Python, SQL, Spark; Snowflake and cloud (AWS/Azure/GCP) experience; pipeline orchestration and data quality expertise.
Python, SQL, Spark, NoSQL, Snowflake, Hive, Hadoop, Airflow, Luigi, Oozie, NiFi, AWS, Azure, Google Cloud, Go
Vistar Media: Operates a programmatic marketplace for digital out-of-home advertising.
4+ YOE4+ years experience building scalable big-data pipelines using Python, Spark, SQL, cloud (GCP), Airflow and Kafka; strong software engineering, data architecture, and geospatial experience preferred.
CoreWeaveNASDAQ: CRWV: Cloud platform providing GPU-accelerated infrastructure for AI workloads.
5+ YOE5+ years building analytical data models and production data systems; expertise in Python/Scala/Rust, advanced SQL, Spark/Flink, lakehouse technologies, and dimensional modeling for OLAP workloads.
Tatari: A platform for buying and measuring TV advertising campaigns.
5+ YOE5+ years building and operating production ETL pipelines; strong Python and SQL skills; Spark/PySpark, Databricks/Delta Lake, Airflow experience; strong data modeling and data quality practices.
Python, SQL, Spark, PySpark, Databricks, Delta Lake, Airflow, ClickHouse
Solidus Labs: Provides market integrity and risk monitoring software for crypto.
8+ YOEBSc in Computer Science; 8+ years in data engineering; 5+ years software engineering with Java, Rust, or Python; deep ClickHouse expertise; experience with Kafka, Spark, Airflow, Kubernetes, Redis, Snowflake, Prometheus/Grafana, and advanced SQL.
NYU Langone Health: Academic medical system providing comprehensive patient care and education.
3+ YOEMaster's in a quantitative field and 3 years' data science/machine learning experience; proficiency in Python or R, scikitlearn, big data tools (Hadoop/Spark), SQL and NoSQL; strong software engineering, deployment, and communication skills.
BarclaysLondon Stock Exchange: BARC: Global bank providing retail, corporate, and investment financial services.
Experience building Java-based data pipelines, data warehouses and lakes; strong skills in Java, Spark, Kafka, Kubernetes, OAuth2/JWT, JUnit, Mockito, GitLab/Bitbucket; experience leading technical teams and ensuring data governance and observability.
Chartmetric: Provides a global music data analytics platform for professionals.
6+ YOE6+ years building production data systems; expert in Python, PostgreSQL, Airflow, Clickhouse, Snowflake, Elasticsearch, Spark, and AWS; Bachelor's in CS/Data Engineering or equivalent experience.
Scottsdale or Chicago or San Francisco or New York City
$97k-$149k/yrHybridFull Time
Early Warning Services: Operates payment and risk solutions for the financial industry.
2+ YOEBachelor's degree typical; 2+ years experience building data engineering tools with Spark and Python, experience with ML toolkits, Hadoop or AWS/S3, and relational databases; strong communication and background/drug screen required.