34 pyspark data engineer jobs at 17 companies in Halfway, MD
1mo
Save
Mark Applied
Hide
Data Engineer
Rockville, Maryland, United States
OnsiteFull Time
Bachelor's in CS/data-related field or equivalent experience; proven data engineering experience with Python, SQL, ETL/ELT, Spark/PySpark, data modeling, data quality, and Git; familiarity with cloud data platforms and biomedical data is a plus.
Python, SQL, Spark, PySpark, Git, Snowflake, Databricks, Posit Connect, R/Shiny, Streamlit, Jupyter, Galaxy, Code Ocean, S3, Postgres, APIs
1w
Save
Mark Applied
Hide
Data Engineer (WDP)
Chantilly or Herndon
OnsiteFull Time
4+ YOEActive TS or TS/SCI clearance required,4+ years data engineering experience,proficiency with Python,SQL,Spark/PySpark,ETL development,cloud data platforms (AWS/Azure),Databricks;Terraform and data quality tooling experience preferred.
Python, SQL, Spark, PySpark, AWS, Azure, Databricks, Databricks Delta Live Tables, Palantir, Snowflake, Terraform, Great Expectations, Airflow, Docker, Kubernetes, NiFi, ElasticSearch
2mo
Save
Mark Applied
Hide
Data Engineer – Senior
Herndon, Virginia, United States
$140k-$180k/yr RemoteFull Time
8+ YOEActive MBI and U.S. citizenship; 8+ years cloud data engineering, 3–5 years Databricks, Databricks and AWS certifications, Python/PySpark/SQL, Delta Lake/medallion architecture, tenant onboarding and federal security compliance (NIST/FISMA/FedRAMP).
Databricks, Unity Catalog, Delta Live Tables, Auto Loader, Genie, Delta Lake, CDC, Python, PySpark, SQL, S3, IAM, VPC, CloudWatch, Glue, Athena, Terraform, GitHub Actions, Microsoft Azure DevOps, Kafka, MSK, dbt, Great Expectations, Splunk, Datadog, Collibra, Alation, AWS Lake Formation
1d
Save
Mark Applied
Hide
Senior Palantir Data Engineer
Herndon or Washington
HybridFull Time
5+ YOEBachelor's degree and 5+ years of data engineering experience, including 3+ years with Palantir Foundry. Requires Python/PySpark, SQL, cloud platforms, and ability to obtain DHS Public Trust clearance.
Pipeline Builder, Code Workbook, Palantir Foundry, Python, PySpark, SQL, TypeScript, AWS, Azure, GCP
1mo
Save
Mark Applied
Hide
Senior AI Data Engineer
Herndon, Virginia, United States
OnsiteFull Time
5+ YOEBachelor's in CS/CE/Data Science, 5+ years data engineering experience, 3+ years Databricks, 5+ years Python/PySpark, Airflow, AWS (S3/Glue/Lambda), MLOps and Generative AI integration experience.
Databricks, Delta Lake, Delta Live Tables, Databricks Workflows, Unity Catalog, Python, PySpark, Apache Airflow, LLMs, S3, Glue, Lambda
1mo
Save
Mark Applied
Hide
Data Engineer
Fairfax or Ashburn
$145k-$160k/yr HybridFull Time
3+ YOEUS citizen able to obtain Public Trust; Bachelor's in STEM required; 3+ years data engineering or related experience; working knowledge of Python and/or SQL; familiarity with Airflow, Spark, or AWS Glue; strong communication skills.
Apache Airflow, AWS Glue, Apache Spark, Kafka, Python, SQL, SharePoint, Confluence, GitLab, AWS, Microsoft Azure, Google Cloud Platform (GCP)
3mo
Save
Mark Applied
Hide
Enterprise Data Engineer
Herndon or Cleveland or California or Colorado or Hawaii or Illinois or Maine or Maryland or Massachusetts or Minnesota or New Jersey or New York or Vermont or Virginia or Washington or District of Columbia
$112k-$222k/yr OnsiteFull Time
3+ YOE3+ years data engineering experience; strong Python and SQL; experience with Palantir or other ETL platforms; data integration, ETL, transformation, and data quality skills; active TS/SCI with Poly required.
Python, SQL, Palantir, AWS, Azure, GCP, Apache Spark, Hadoop, Apache Airflow, Prefect
1mo
Save
Mark Applied
Hide
Senior AI Data Engineer
Herndon, Virginia, United States
OnsiteFull Time
5+ YOE5+ years building large-scale data platforms with 3+ years Databricks; 5+ years Python/PySpark and Airflow experience; 3+ years with AWS (S3, Glue, Lambda); BA/BS in CS or related; Databricks/AWS certifications preferred.
Databricks, Delta Lake, Delta Live Tables (DLT), Databricks Workflows, Unity Catalog, Python, PySpark, Apache Airflow, LLMs, S3, Glue, Lambda
1mo
Save
Mark Applied
Hide
Senior Data Engineer
Herndon, Virginia, United States
$150k-$175k/yr HybridFull Time
8+ YOEUS citizen with active Secret clearance; B.S. in Computer Science (or equivalent) and 8+ years experience; strong Python, AWS, SQL/Trino, Apache Spark, dbt, and Kubernetes (Rancher) experience; DevSecOps and TDD practices.
Python, FastAPI, Django, Flask, JavaScript, Node.js, Express, Vue, Svelte, React, Java, Spring Boot, AWS, S3, RDS, EMR, Trino, Apache Spark, dbt, Rancher Kubernetes, Rancher RKE2, AWS EKS, Apache Airflow, HL7, FHIR, Mirth Connect, Great Expectations, OpenAPI, ORM
4w
Save
Mark Applied
Hide
Data Analytics Engineer
Ashburn, Virginia, United States
HybridFull Time
1+ YOEU.S. security clearance (DoD Top Secret/DHS EOD/CBP Full BI/IT-I or SSBI), Bachelor’s in computer science or related, 1–3 years SQL and BI experience, Databricks/AWS exposure, Python (preferred).
SAP Business Objects, Oracle, Databricks, S3, EMR, Redshift, Glue, PowerBI, Tableau, Qlik, Python, Pandas, PySpark, SQL
1mo
Save
Mark Applied
Hide
Enterprise Data Engineer, TS/SCI w/ Poly Chantilly, VA
Chantilly or Herndon
OnsiteFull Time
3+ YOE3+ years data engineering experience; strong Python and SQL skills; experience with Palantir or ETL platforms; data integration, transformation, validation, and monitoring; active TS/SCI with FSP required to start.
Python, SQL, Palantir, Apache Spark, Hadoop, AWS, Azure, GCP, Apache Airflow, Prefect
3mo
Save
Mark Applied
Hide
Big Data Analyst (TS/SCI)
Herndon or Springfield
$118k-$196k/yr OnsiteFull Time
5+ YOEActive TS/SCI clearance required; 5+ years as data analyst/scientist; bachelor's degree or equivalent experience; strong data evaluation, data engineering, and communication skills; proficient with Python, SQL, Spark/PySpark; on-site in VA.
Python, SQL, Spark, PySpark, Elasticsearch/Kibana, Gitlab, Jupyter Notebooks, Apache Zeppelin
3mo
Save
Mark Applied
Hide
Apache Spark Developer
Herndon or Springfield or St. Louis or Aurora
$150k-$225k/yr OnsiteFull Time
5+ YOETS/SCI eligibility preferred, Bachelor\u0002s plus 5+ years in data engineering or Spark development, strong Spark and PySpark experience, Python, large-scale data processing, performance tuning, Kubernetes/Docker familiarity.
Apache Spark, PySpark, Scala, Delta Lake, Parquet, Structured Streaming, Kubernetes, Docker, S3, AWS, GCP, Azure, Git, Jenkins, Python, Bash, Terraform, Iceberg
1mo
Save
Mark Applied
Hide
HR AI/ML Engineer II
Herndon, Virginia, United States
$135k-$150k/yr OnsiteFull Time
3+ YOEDesign and deploy AI/ML solutions for HR, build data pipelines, strong Python and ML library experience, cloud and data engineering experience, 3+ years AI/ML or data engineering experience, ability to travel up to 10%.
Google Cloud Platform (GCP), AWS, Vertex AI, AutoML, AI Infrastructure, S3, Data Factory, BigQuery, Python, TensorFlow, PyTorch, Scikit-learn, Spark, Airflow, Databricks, PySpark, Scala, Git
3w
Save
Mark Applied
Hide
Software Development Engineer II, Amazon Security Data Engineering
Herndon, Virginia, United States
$144k-$194k/yr OnsiteFull Time
3+ YOE3+ years professional software development and system design experience; bachelor’s in CS/Engineering/Mathematics; proficiency in Java/Scala/Python; experience with distributed systems and big data engines (Iceberg, Spark, Trino, Flink).
Java, Scala, Python, Apache Iceberg, Apache Spark, Trino, Flink, AWS Glue, Parquet, C#, C++, Perl
1mo
Save
Mark Applied
Hide
Cloud Software Engineer II
Herndon or Springfield
OnsiteFull Time
7+ YOETS/SCI with Poly required; 7+ years software engineering; 5+ years Python and JS/TS; AWS CDK and cloud-native services; microservices, data engineering, ETL, DevSecOps, observability experience.
AWS, AWS CDK, Python, FastAPI, Flask, Django, Pandas, NumPy, Apache Spark, JavaScript, TypeScript, Node.js, npm, React, Vue.js, Angular, Lambda, API Gateway, EventBridge, SQS, SNS, RDS, DynamoDB, Redshift, S3, Athena, ECS, Fargate, EKS, Glue, EMR, Kinesis, Step Functions, SageMaker, Databricks, SonarQube, Snyk, Checkmarx, CloudWatch, DataDog, New Relic, Prometheus, Grafana
2mo
Save
Mark Applied
Hide
Software Engineer
Herndon or Potomac
OnsiteFull Time
Full stack developer with backend focus; Solr/Elasticsearch, Lucene, big data tools, Java backend, and security clearance requirements.
Solr, Elasticsearch, Lucene, Hadoop, PySpark, Spark, HBase, Java, AWS, Jira, GitHub
2w
Save
Mark Applied
Hide
Database Developer
Ashburn, Virginia, United States
$100k-$125k/yr HybridFull Time
4+ YOEBachelor's degree plus 4+ years in database development, strong SQL, PostgreSQL, Python, Git, AWS, ETL, data profiling, performance tuning, and strong communication skills.
PostgreSQL, Python, Git, AWS, CloudWatch, Kibana, Dynatrace, SQL, Oracle, DB2, SQLServer, MySQL, Teradata, Databricks, PySpark, Liquibase, DMS, Jira
2mo
Save
Mark Applied
Hide
Lead Specialist, Federal Artificial Intelligence (AI) Engineering
San Francisco or Ashburn or McLean
OnsiteFull Time
5+ YOE5+ years AI/IT/data science experience, bachelor’s degree required, experience with NLP/unstructured data, distributed computing and cloud platforms, production-level coding, and eligible for U.S. government security clearance.
Hadoop, MapReduce, HDFS, Spark, PySpark, Hive, Impala, H-Base, MongoDB, Casandra, Teradata, Netezza, Redshift, AWS, Microsoft Azure, GCP, Web/RESTful, HTML5, JavaScript, Angular, React, Redux, Linux, EMR
1w
Save
Mark Applied
Hide
Ontology Subject Matter Expert (TS/SCI)
Herndon, Virginia, United States
$165k-$242k/yr OnsiteFull Time
12+ YOEU.S. citizen with active TS/SCI+CI polygraph, 12+ years experience with advanced degree (or 17+ with bachelor's), enterprise data engineering, ontology/semantic web, cloud/container platforms, and technical leadership.
RDF, OWL, SPARQL, Kubernetes, OpenShift, Apache Spark, Kafka, NiFi, Airflow
Explore Jobs