HiringCafe
Join our community

216 pyspark developer jobs at 119 companies in New York

3mo
Save
Mark Applied
Hide
C++/Pyspark Developer with Mortgage Risk (438808)
New York or Manhattan
$90k-$110k/yr OnsiteFull Time
Capgemini
CapgeminiEuronext Paris: CAP: Provides global IT consulting and digital transformation services.
7+ YOE7+ years C++ development; strong Python and PySpark data engineering; SQL, Databricks, Snowflake; grid computing and SDLC tools (Jira, HP Quality Center); strong communication and risk-modeling experience.
C++, Python, PySpark, Databricks, Snowflake, Jira, HP Quality Center
Job PostingView all
1w
Save
Mark Applied
Hide
Sr. Associate, Date Engineer - PySpark
Atlanta or New York City or Philadelphia or Dallas
OnsiteFull Time
KPMG
KPMG: Global professional services network providing audit, tax, and advisory.
5+ YOE5+ Mgmt5+ years leading data science/engineering teams, Master’s degree in CS/DS/Statistics/Engineering, hands-on Python, PySpark, SQL, Databricks, Snowflake, experience with cloud platforms, ability to travel, U.S. work authorization required.
Python, PySpark, SQL, Databricks, Snowflake, Linux, AWS, GCP, Azure, R
Job PostingView all
2w
Save
Mark Applied
Hide
Sr. Software Engineer - PySpark
Scottsdale or San Francisco or Chicago or New York City
$118k-$152k/yr HybridFull Time
Early Warning Services
Early Warning Services: Operates payment and risk solutions for the financial industry.
5+ YOEBachelor's-level education typical; minimum 5 years software engineering experience; strong with Git, Spring/Spring Boot, SQL, REST, Docker/Kubernetes, Kafka, automated testing and DevOps practices.
Git, Git flow, Spring, Spring Boot, SQL, Confluence, CI/CD, Docker, Kubernetes, Kafka, Hadoop, Hive, XML, JSON, YAML, Redis, HBASE, AeroSpike, REST
Job PostingView all
3d
Save
Mark Applied
Hide
Software Engineer II
Seattle or New York City
$123k-$165k/yr HybridFull Time
The Walt Disney Company
The Walt Disney CompanyNYSE: DIS: Produces movies, operates theme parks, and provides streaming services.
3+ YOERequires 3+ years of backend development, AI or data-driven applications, scalable APIs, foundation models, ML frameworks, PySpark or Pandas, AWS, Docker, Git, and a bachelor's degree or equivalent experience.
Claude Opus, Claude Sonnet, GPT-4, Amazon Bedrock, FlaskAPI, Flask, LangChain, LangGraph, PyTorch, TensorFlow, PySpark, Pandas, AWS, Docker, Git, Cursor, Datadog, Grafana, Conviva, MongoDB
Job PostingView all
3w
Save
Mark Applied
Hide
Spark Engineer
Jersey City or New York City
OnsiteFull Time
Infosys
InfosysNYSE: INFY: Provides IT consulting, software development, and business outsourcing services.
Experience with Spark/Hadoop ecosystem, Kafka, HBase/Hive/Impala, Python/SQL/PySpark/Shell; bachelor's degree or equivalent experience; ability to work full SDLC; US work authorization required.
Spark, Hadoop, HDFS, YARN, Confluent Kafka, HBase, Hive, Impala, NoSQL, PySpark, SQL, Python, Shell
Job PostingView all
1mo
Save
Mark Applied
Hide
Senior Software Engineer - Pricing & Promotions Analytics
New York City, New York, United States
$111k-$222k/yr HybridFull Time
CVS Health
CVS HealthNYSE: CVS: Provides retail pharmacy, health insurance, and pharmacy benefit management services.
3+ YOE3+ years software development experience building cloud-based customer-facing analytic solutions; experience with Palantir Foundry or similar, Python/PySpark, SQL, LLM modeling, cloud platforms (Azure/GCP/AWS); BA/BS required; strong communication and collaboration skills.
Palantir Foundry, Workshop, Pipeline Builder, TypeScript, Python, PySpark, SQL, LLM, Azure, GCP, AWS
Job PostingView all
2mo
Save
Mark Applied
Hide
Principal Backend Engineer
New York, New York, United States
$135k-$217k/yr OnsiteFull Time
Publicis Groupe
Publicis GroupeEuronext Paris: PUB: Global advertising and digital transformation agency holding.
10+ YOE10+ years backend or distributed systems experience; expert Python backend; Databricks/PySpark; AWS serverless; multi-tenant SaaS design; leadership without people management.
Python, SQL, Databricks, PySpark, AWS, API Design, DynamoDB, S3, Lambda, IAM, CloudFormation, SAM
Job PostingView all
1mo
Save
Mark Applied
Hide
Staff Software Engineer - DSM
Spain or Brazil or Stockholm or London or New York City or New Orleans or Singapore or Tokyo or Sydney
RemoteFull Time
Cint
CintNasdaq Stockholm: CINT: Operates a digital platform for programmatic market research surveys.
10+ YOE10+ years backend engineering experience; strong algorithms and architecture knowledge; experience with high-throughput, low-latency systems; familiarity with Java, Python/PySpark, Spark, Kafka, Docker, SQL/NoSQL.
Java, Python, PySpark, Spark, Delta Lake, Parquet, Kafka, Avro, Docker, SQL, NoSQL, REST APIs, Pandas, Spring, AWS, GCP
Job PostingView all
1d
Save
Mark Applied
Hide
Palantir Forward Deployed Engineer
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$54k-$235k/yr HybridFull Time
Accenture
AccentureNYSE: ACN: Global professional services firm providing consulting and technology solutions.
3+ YOERequires 3+ years in data engineering, cloud platforms, and Python, PySpark, or Java; 1+ year with Palantir Foundry and AIP; bachelor's degree or equivalent experience.
Palantir Foundry, Palantir Gotham, Palantir Artificial Intelligence Platform (AIP), Python, TypeScript, PySpark, Java, Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), Git, GitHub, GitLab, Jenkins, JavaScript
Job PostingView all
3w
Save
Mark Applied
Hide
Software Engineer AI/ML Ops
Pleasanton or New York City or Los Angeles
$145k-$182k/yr HybridFull Time
BlackLine
BlackLineNASDAQ: BL: Provides cloud-based financial close and accounting automation software.
2+ YOE2+ years in Python/Java/Scala, experience building PySpark ETL, ML/LLM production pipelines, cloud (GCP/AWS/Azure), ML frameworks, orchestration, containerization, and observability.
PySpark, FiveTran, Plaid, TensorFlow, PyTorch, scikit-learn, Airflow, Kubeflow, Vertex AI, MLflow, GCP, AWS, Azure, LangChain, LangGraph, ADK, Prometheus, Grafana, Newrelic, Docker, Kubernetes, Bash, Python, Java, Scala
Job PostingView all
2mo
Save
Mark Applied
Hide
Lead Software Engineer - Parametric
New York, New York, United States
$115k-$225k/yr HybridFull Time
Morgan Stanley
Morgan StanleyNYSE: MS: Provides global investment banking, wealth management, and advisory services.
8+ YOE8+ years software engineering experience, Bachelor’s in CS or related, expert Python, 5+ years cloud-native AWS (Lambda, Step Functions, Batch, S3, Athena, Glue), Kafka, SQL/NoSQL, PySpark, Terraform, Snowflake, leadership and Agile experience.
Python, Apache Kafka, PySpark, AWS Lambda, AWS Step Functions, AWS Batch, S3, AWS Athena, AWS Glue, DynamoDB, Terraform, Snowflake, SQL, C#, .NET, CI/CD, AI/GenAI
Job PostingView all
1mo
Save
Mark Applied
Hide
Staff Software Engineer Data - DC Tech Lead
Toronto or Alabama or Arkansas or California or Colorado or Florida or Georgia or Illinois or Kentucky or Massachusetts or Michigan or Montana or Missouri or Nevada or New Jersey or New York or North Carolina or Oregon or Pennsylvania or Texas or Washington or Utah or Virginia or Wisconsin
$148k-$242k/yr RemoteFull Time
Afresh
Afresh: AI-powered inventory management for grocery store fresh food departments.
Design and maintain large-scale ETLs using PySpark and DBT; proficiency in Python, SQL, Databricks or Snowflake; 2+ years technical lead experience; strong communication and mentoring skills.
PySpark, DBT, Python, SQL, Databricks, Snowflake
Job PostingView all
1mo
Save
Mark Applied
Hide
Palantir Forward Deployed Engineer
Chicago or Milwaukee or Dallas or Columbus or Kirkland or Cincinnati or New York or Cleveland or Oklahoma City or Austin or Albany or St. Petersburg or Hartford or Pittsburgh or St. Louis or Miami or Sacramento or Raleigh or Minneapolis or Mountain View or Scottsdale or San Francisco or Morristown or Denver or Boston or Philadelphia or Des Moines or Overland Park or Los Angeles or Charlotte or Walnut Creek or Carmel or Seattle or Houston or Arlington or Atlanta or Redmond or Bentonville or Beaverton or Nashville or Detroit or San Diego
$54k-$235k/yr HybridFull Time
Accenture
AccentureNYSE: ACN: Global provider of management consulting and technology services.
3+ YOE3+ years data engineering; 1+ year Palantir Foundry experience; 1+ year prompt engineering/AIP; 3+ years cloud (AWS/Azure/GCP); 3+ years Python/PySpark/Java; Bachelor's in CS/Engineering or equivalent; strong communication.
Palantir Foundry, Palantir Gotham, Artificial Intelligence Platform (AIP), Python, TypeScript, PySpark, Java, AWS, Azure, GCP, Git, GitHub, GitLab, Jenkins
Job PostingView all
1w
Save
Mark Applied
Hide
Senior Staff Machine Learning Engineer
New York City, New York, United States
$240k-$250k/yr HybridFull Time
Grubhub
Grubhub: A food ordering and delivery marketplace that connects diners with restaurants and provides restaurant technology.
8+ YOEMS/PhD or equivalent; 8+ years building ML systems and 3+ at staff scope. Requires scalable recommendation, ranking, retrieval, deep learning, Python, PySpark, Hive/SQL, cloud ML, and production deployment expertise.
TensorFlow, PyTorch, GPUs, Large Language Models, AWS, SageMaker, PySpark, Hive, SQL, Python, A/B tests
Job PostingView all
2mo
Save
Mark Applied
Hide
Vice President, Full-Stack Engineer
New York, New York, United States
$83k-$209k/yr OnsiteFull Time
BNY
BNYNYSE: BK: Global institution managing and servicing financial assets worldwide.
Bachelor's degree in CS/Engineering, strong experience with Python, PySpark, Java, SQL Server/Oracle/MongoDB, Linux/Shell and PowerShell, CI/CD and test automation, and deep knowledge of financial risk concepts and fixed income products.
Python, PySpark, Java, SQL Server, Oracle, MongoDB, Linux, Shell, PowerShell, CI/CD, Quant Model, AI/ML
Job PostingView all
1mo
Save
Mark Applied
Hide
Data Engineer
New York or Los Angeles or San Francisco
$140k-$175k/yr HybridFull Time
Regard
Regard: A that builds AI-powered software to improve clinical documentation and advance patient care.
3+ YOEBS or equivalent, 3+ years data engineering experience, Python and SQL proficiency, experience with PySpark and distributed data processing, AWS experience preferred, LLM-assisted development familiarity, and willingness to participate in on-call support.
S3, Apache Iceberg, EMR, PySpark, Dagster, Kubernetes, ClickHouse, PostgreSQL, FastAPI, Metabase, Python, SQL, AWS, AWS Athena, EHR, OMOP CDM, HIPAA, LLM
Job PostingView all
10h
Save
Mark Applied
Hide
Compliance, Machine Learning Engineer, New York, Vice President
New York City, New York, United States
$130k-$250k/yr OnsiteFull Time
Goldman Sachs
Goldman SachsNYSE: GS: Global investment banking, securities, and investment management firm.
10+ YOEBachelor’s or master’s in computer science or related field; 10+ years building scalable machine learning systems; strong coding and systems fundamentals; expertise in Python, PySpark, distributed technologies, and ML toolkits.
Python, PySpark, Scala, Iceberg, HDFS, Avro, Parquet, AWS, GCP, TensorFlow, PyTorch, Scikit-learn, Hugging Face
Job PostingView all
3mo
Save
Mark Applied
Hide
Machine Learning Software Engineer Intern - Winter 2027
San Francisco or New York City
HybridInternship, Temporary
Rippling
Rippling: Unified platform managing workforce HR, IT, and finance operations
Currently enrolled in an M.Sc. or Ph.D. in computer science or related field; strong backend programming skills (Python, PySpark, PyTorch); ML experience, LLM familiarity preferred; excellent communication and learning mindset.
Python, PySpark, PyTorch, LLMs
Job PostingView all
4d
Save
Mark Applied
Hide
Data Engineer - Capital Markets
New York City, New York, United States
$97k-$156k/yr HybridFull Time
SMBC Group
SMBC GroupNew York Stock Exchange: SMFG: Provides global banking, investment, securities, and consumer finance services.
4+ YOERequires 4+ years designing and developing on Databricks, 3+ years in AWS, advanced Python/PySpark and SQL, ETL/ELT, Spark, Delta Lake, streaming, CI/CD, and Terraform experience.
Databricks, Databricks Jobs, Delta Live Tables (DLT), Amazon S3, AWS Glue, AWS IAM, Apache Spark, Delta Lake, Unity Catalog, Databricks Asset Bundles (DABs), Terraform, GitHub Actions, GitLab CI/CD, AWS CodePipeline, Python, PySpark, SQL, Amazon MSK, Kafka
Job PostingView all
1w
Save
Mark Applied
Hide
Software Engineer
San Francisco or Seattle or Maryland or New Jersey or New York or Alaska or Hawaii or Maine or Mississippi or North Dakota or South Dakota or Vermont or West Virginia or Wyoming
$118k-$183k/yr HybridFull Time
DocuSign
DocuSignNASDAQ: DOCU: Provides electronic signature and agreement management software solutions.
3+ YOEBachelor's degree in a relevant field, 3+ years of software development, and experience with search engines, distributed processing, public cloud services, and CI/CD tools.
C#, Java, C++, Go, Elasticsearch, Solr, Apache Spark, Kafka, Azure, AWS, GCP, Git, Jenkins, Azure DevOps, Docker, Kubernetes
Job PostingView all

Explore Jobs

Expand Your Job Search