Company Background
Our client is a leading life insurance company and part of a global financial services group. Founded in 1896 and headquartered in North America, the company offers a wide range of products, including annuities and life insurance, through a nationwide network of agents.
Project Description
The project involves a production system for managing clients and the life insurance programs in which they are enrolled. The specialist will be responsible for supporting the existing system and developing new features.
Technologies
- Databricks, Unity Catalog, Delta Lake, Databricks Genie
- Advanced SQL
- Python
- Microsoft Azure
What You'll Do
- Collaborate with a team of three engineers to develop and support data solutions in Databricks;
- Develop and maintain data processing workflows using Databricks, SQL, Python, and Delta Lake;
- Work with Unity Catalog to manage data access permissions, data lineage, and governance;
- Support the existing production system and participate in the implementation of new features and enhancements;
- Investigate and troubleshoot issues in Databricks data workflows and production processes;
- Collaborate with team members and stakeholders to deliver reliable data engineering solutions;
Job Requirements
- 3+ years of professional experience in data engineering;
- Strong communication and collaboration skills, with the ability to work effectively with technical and non-technical stakeholders;
- Hands-on experience designing and developing data solutions in Databricks, including Databricks architecture, Unity Catalog, Delta tables, data lineage, access permissions, Databricks Genie, and notebooks;
- Advanced SQL skills, including writing complex queries and performing data transformations;
- Proficiency in Python for data processing, automation, and development of data engineering solutions;
- Professional experience working with Microsoft Azure cloud services and data platforms;
Nice To Have
- Experience using Databricks AI capabilities, including Agent Bricks, and implementing Generative AI and RAG solutions in Databricks;
- Experience working with Azure data services, including Azure Synapse, Synapse pipelines, Azure Data Factory, and Azure SQL pools;
- Experience setting up CI/CD processes for Databricks solutions using Azure DevOps, Databricks Asset Bundles, and Infrastructure as Code practices;
- Experience with performance tuning for Databricks workloads, SQL queries, and data processing pipelines;
- Databricks Certified Data Engineer Professional certification;
- Databricks Certified Generative AI Engineer Associate certification;
What Do We Offer
The global benefits package includes:
- Technical and non-technical training for professional and personal growth;
- Internal conferences and meetups to learn from industry experts;
- Support and mentorship from an experienced employee to help you professional grow and development;
- Health insurance;
- Sports activities to promote a healthy lifestyle;
- Flexible work options, including remote and hybrid opportunities;
- Referral program for bringing in new talent;
- Work anniversary program and additional vacation days.
