Overview:
As a Data Engineer, you will play a critical role in transforming raw data into valuable insights that drive our business decisions. You will design, develop, and maintain data pipelines and infrastructure across hybrid cloud environments, while also building robust reporting and visualization solutions.
Responsibilities:
- Data Pipeline Development: Design, build, and maintain scalable data pipelines to extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files) into data warehouses or data lakes.
- Hybrid Cloud Infrastructure: Manage and optimize data infrastructure across hybrid cloud environments, leveraging cloud-native services and on-premises resources.
- Data Quality: Ensure data quality through implementation of data validation, cleansing, and standardization processes.
- Reporting and Visualization: Develop interactive reports and dashboards using tools like Power BI, Tableau, or Looker to provide actionable insights to stakeholders.
- Data Governance: Adhere to data governance policies and procedures, including data security, privacy, and compliance regulations.
- Data Modeling: Design and implement data models (e.g., dimensional, normalized) to optimize data storage and retrieval.
- Automation: Automate data pipelines and processes using scripting languages (e.g., Python, SQL) and automation tools.
- Collaboration: Work closely with data analysts, scientists, and business users to understand their requirements and deliver relevant data solutions.
