Job Summary
We are looking for a Data Engineer to build, maintain, and optimize scalable data pipelines and datasets that support analytics, operational reporting, marketing use cases, and AI-enabled solutions.The ideal candidate has strong hands-on experience with Databricks, Apache Spark, Python, and modern ETL/orchestration tools such as Azure Data Factory (ADF) or Airflow. This role is best suited for someone who enjoys solving data engineering problems, working with large datasets, improving performance, and partnering with cross-functional teams to deliver reliable and scalable data solutions.
Experience working with agentic AI or AI-enabled data workflows is a plus, and familiarity with the MarTech ecosystem is highly desirable.
Job Description
Key Responsibilities
- Develop, maintain, and enhance batch and/or near-real-time data pipelines using Databricks, Spark, Python, and ADF or Airflow.
- Build data workflows that ingest, transform, validate, and publish large-scale datasets for analytics, operational, and downstream platform consumption.
- Work with product managers, analysts, data consumers, and engineering teams to understand requirements and translate them into effective data solutions.
- Optimize data processing jobs for performance, reliability, scalability, and cost efficiency.
- Support the design and implementation of data models, curated datasets, and transformation logic for business and technical use cases.
- Troubleshoot data issues across ingestion, transformation, orchestration, and delivery layers.
- Ensure data quality, consistency, and observability through validation, monitoring, logging, and testing practices.
- Contribute to modernization efforts such as migrating legacy workflows to cloud-native or lakehouse-based platforms.
- Support secure and compliant handling of enterprise and customer-related data.
- Help enable AI and automation use cases by preparing reliable, scalable, and high-quality data assets.
- Collaborate with teams supporting analytics, personalization, customer platforms, and operational systems.
Required Qualifications
- Bachelor’s degree in Computer Science, Engineering, Mathematics, Information Systems, or a related technical field (or equivalent practical experience).
- 3+ years of experience in data engineering, software engineering, or related technical roles.
- Hands-on experience with Databricks in a production or enterprise data environment.
- Strong programming skills in Python and experience using Apache Spark for large-scale data processing.
- Experience with ETL and orchestration tools such as Azure Data Factory (ADF), Airflow, or equivalent platforms.
- Experience building and supporting pipelines for large and complex datasets.
- Experience with data optimization, including tuning jobs, improving pipeline performance, and designing efficient processing patterns.
- Strong SQL skills and experience with data transformation, data modeling, and schema design.
- Familiarity with modern cloud data platforms and lakehouse / warehouse concepts.
- Strong problem-solving and debugging skills in data-intensive environments.
- Good communication skills and the ability to work effectively with cross-functional teams.
