HiringCafe
Save Job
Posted 2w ago

Sr Data Engineer - AWS Services + Python + SQL

@ Tech Mahindra
View All Jobs
Pune, Maharashtra, India
OnsiteFull Time
Responsibilities:executing migration, developing pipelines, validating data
Requirements Summary:7+ years data engineering experience with cloud platforms; Bachelor's degree; strong SQL, Python, PySpark; Databricks or Snowflake; AWS services; Airflow and GitLab experience; data validation and pipeline development.
Technical Tools Mentioned:SQL, Python, PySpark, Apache Spark, Databricks, Delta, DLT, Autoloader, Unity Catalog, Snowflake, Snowpark, Snowpipe, Streams, Tasks, AWS, S3, Redshift, Athena, Glue, EMR, Lambda, Apache Airflow, GitLab, Control M, Qlik Replication, AWS DMS, Shell, Linux/Unix
Save
Mark Applied
Hide Job
Report & Hide
Job Description

Job Description

  • Skill Set : Data engineering
  • Total Experience : 7.00 to 10.00 Years
  • No of Openings : 1
  • Job Post Date : 26/06/2026
  • Job Expiry Date : 31/07/2026
  • Domain : IT
  • Location : PUNE [India]
  • Job Reference No : 4090891

A Bachelor’s or Higher Degree is the minimum entry required for the position

Job Summary

Senior Data Engineer

Role: Senior Data Engineer ‿ Migration Execution

Location: Offshore (India)

Experience: 7 10 years in data engineering with cloud platforms

Role Summary: Execute priority workload migrations across all 3 tracks (L&S, Refactor, New Build) ‿ building production grade pipelines on Databricks and Snowflake, running dual run validations, and remediating technical debt across schemas, naming, and lineage.

Key Responsibilities:

  • Execute automated Lift & Shift migrations ‿ schema mapping, Iceberg table creation, pipeline generation for Bronze/Silver tables
  • Develop and refactor ETL/ELT pipelines ‿ decompose monolithic PSQL/Glue jobs into modular PySpark/Snowpark patterns
  • Review, validate, and refine AI assisted code conversions (PySpark/Snowpark); label AI generated lines in code comments per EchoStar Addendum
  • Configure and execute dual run validation scripts ‿ row count, hash based integrity, business rule comparisons
  • Rationalize schemas, naming conventions, and lineage for AI Factory consumption patterns and Unity Catalog / Horizon Catalog governance
  • Develop and deploy Airflow DAGs replacing Control M orchestration
  • Produce before/after metrics (complexity, runtime, cost) for Technical Debt Remediation Log
  • Create operations runbooks, SOPs, and procedural guides for each migrated workload
  • Support EchoStar FTE shadowing and reverse shadowing during KT phases

Must Have Skills:

  • Strong proficiency in SQL, Python, PySpark, Apache Spark
  • Hands on experience with Databricks (Delta, DLT, Autoloader, Unity Catalog) and/or Snowflake (Snowpark, Snowpipe, Streams, Tasks)
  • Experience with AWS services ‿ S3, Redshift, Athena, Glue, EMR, Lambda
  • Hands on with Apache Airflow DAG development
  • Experience with GitLab for version control and CI/CD
  • Strong SQL performance tuning and optimization skills
  • Experience with data quality validation and reconciliation

Nice to Have:

  • Experience migrating from Redshift/Athena to Databricks or Snowflake
  • Familiarity with Control M and Control M â¿¿ Airflow migration
  • Experience with Qlik Replication / AWS DMS
  • Shell scripting (Linux/Unix)
  • AWS or Databricks certifications