JOB SUMMARY
We are seeking an experienced Data Engineer with strong hands-on Databricks expertise to support the design, development, and optimization of enterprise data solutions. The successful candidate will have deep experience building and maintaining ELT pipelines, working with large-scale data platforms, and applying modern data architecture principles. This role requires alignment to UK business hours and on-site presence at our delivery center.
JOB RESPONSIBILITIES
- Design, develop, and maintain ELT pipelines using Databricks and related technologies.
- Build and optimize large-scale data processing workflows using PySpark and SQL.
- Develop and manage data models to support analytics, reporting, and business intelligence requirements.
- Implement and support data Lakehouse architectures and data warehousing solutions.
- Collaborate with data analysts, architects, and business stakeholders to understand and deliver on data requirements.
- Ensure data quality, integrity, and governance across all data platforms.
- Monitor and troubleshoot data workflows, pipeline performance, and platform issues.
- Contribute to continuous improvement of data engineering standards and practices.
- Support and work during UK business hours as required.
REQUIRED TECHNICAL SKILLSET
- Strong hands-on experience with Databricks (Delta Lake, Databricks Workflows, Unity Catalog).
- Proven experience building and supporting ELT pipelines at scale.
- Strong proficiency in PySpark and SQL.
- Experience with data modeling techniques and best practices.
- Good understanding of data Lakehouse architecture and data warehousing concepts.
- Experience with version control tools (e.g., Git).
- Familiarity with cloud platforms (Azure preferred; AWS or GCP considered).
PREFERRED EXPERIENCE
- Experience with orchestration tools such as Apache Airflow or Azure Data Factory.
- Exposure to data governance frameworks and tools (e.g., Unity Catalog, Microsoft Purview).
- Experience in enterprise consulting or regulated industry environments.
- Experience working with globally distributed teams across different time zones.
CERTIFICATIONS PREFERRED
- Databricks Certified Associate Developer for Apache Spark.
- Databricks Certified Data Engineer Associate or Professional.
- Microsoft Certified: Azure Data Engineer Associate (DP-203) or equivalent.
- Additional data or cloud certifications are a plus.
