HiringCafe
Deloitte
Posted 1w ago

Manager | GCP Data Engineer | Ahmedabad | Engineering | Data Modernization & Migration

Deloitte
deloitte.comView all open roles
Ahmedabad, Gujarat, India
OnsiteFull Time
Responsibilities
  • designing solutions
  • optimizing workloads
  • leading teams
Requirements
  • Requires 8–15+ years in data engineering with strong GCP experience
  • Python, Spark
  • Dataproc
  • BigQuery
  • Airflow
  • Cloud architecture
  • Performance optimization
  • Stakeholder management, and team leadership
Technical tools mentioned
Google Cloud Platform (GCP)Cloud RunCloud FunctionsPythonCloud SQL PostgreSQLPub/SubGoogle Cloud Storage (GCS)BigQueryCloud ComposerManaged AirflowAirflowDatabricksREST APIsSparkDataprocYARNSpark UI

Job description

Job Title: 
Manager | GCP Data Engineer | Ahmedabad | Engineering | Data Modernization & Migration

Role Summary

We are looking for a GCP Data Engineering Manager / AD with 8 to 15+ years of experience to lead the delivery and optimization of cloud-based data platforms. The role combines strong expertise in GCP data services with hands-on performance engineering for large-scale Spark and Dataproc workloads.

Key Responsibilities

GCP Data Engineering and Platform Delivery

  • Design and deliver data solutions using Cloud Run, Cloud Run Functions with Python, Cloud SQL PostgreSQL, Pub/Sub, GCS and BigQuery.
  • Build and manage orchestration workflows using Cloud Composer / Managed Airflow.
  • Drive Airflow and Composer optimization, including FinOps, capacity planning and cost governance.
  • Support Databricks-based data processing and integration.
  • Design and integrate REST APIs across cloud applications and data platforms.
  • Provide technical leadership, architecture guidance and delivery oversight to engineering teams.

Performance Engineering and Dataproc Optimization

  • Execute workload test waves and enhance Python-based execution templates.
  • Troubleshoot Spark, YARN and Dataproc failures using Spark UI and platform diagnostics.
  • Analyze approximately 60 workload profiles and capture cost-performance benchmarks.
  • Define N1/N2 mapping strategies and translate custom machine requirements into standard GCP machine types.
  • Recommend VM sizing, schema design and performance improvements for Big Data workloads.

Required Skills

  • 8 to 15 years in Data Engineering, with strong experience on GCP.
  • Python, Spark, Dataproc, GCS, BigQuery and Cloud Data Architecture.
  • Cloud Run, Cloud Functions, Cloud SQL PostgreSQL and Pub/Sub.
  • Cloud Composer / Airflow administration, optimization and FinOps.
  • Spark UI debugging, YARN diagnostics and workload benchmarking.
  • Strong stakeholder management, team leadership and solution design capabilities.


About Deloitte

Professional services firm providing audit, consulting, and advisory services.

Year founded
1845
Employees
460000
Industries
Professional ServicesManagement Consulting
Activities
Audit and assuranceManagement consultingTax advisory servicesRisk advisory servicesFinancial advisory services
Organization type
Private
Latest investment
Raised $35.40M Debt Financing (2023) — led by RBL Bank
Subsidiaries
Deloitte ConsultingDeloitte DigitalDeloitte TaxDeloitte Audit
Headquarters
GB

Similar jobs

Data Engineer roles near Ahmedabad, Gujarat
1w
Save
Mark Applied
Hide
Data Engineer
Ahmedabad, Gujarat, India
OnsiteFull Time
Center for Computational Technologies
Center for Computational Technologies: Develops specialized software for engineering, CAD, and CFD applications.
4+ YOERequires 4–7 years of experience with Python, SQL, Azure or GCP, ETL pipelines, APIs, and time-series data analysis; renewable energy systems knowledge is preferred.
Python, SQL, Azure, GCP, ETL, APIs
Job PostingView all
1w
Save
Mark Applied
Hide
Databricks Data Engineer
Ahmedabad, Gujarat, India
OnsiteFull Time
Tata Consultancy Services
Tata Consultancy ServicesNational Stock Exchange of India: TCS: Global provider of IT services, consulting, and business solutions.
8+ YOERequires 8+ years of data engineering experience, 4+ years with Databricks, expertise in Spark, Data Lakehouse and Medallion Architecture, cloud experience with AWS or Azure, and a bachelor's engineering degree.
Databricks, PySpark, Delta Lake, Amazon Web Services (AWS), Microsoft Azure, Spark, Data Lakehouse, Medallion Architecture
Job PostingView all
2w
Save
Mark Applied
Hide
Data Engineer (ETL, ADF)
Ahmedabad or Pune or Indore
OnsiteFull Time
Arrow Electronics
Arrow ElectronicsNYSE: ARW: Distributes electronic components and provides enterprise computing solutions.
5+ YOERequires 5 years of data engineering experience with Azure Data Factory, SQL, stored procedures, ETL, data warehouses, Power BI, performance tuning, Agile methods, communication, and data analysis skills.
Azure Data Factory, SQL, Microsoft Power BI, SQL Server, SQL Stored Procedures
Job PostingView all
2w
Save
Mark Applied
Hide
Data Engineer
Hyderabad or Ahmedabad or Chennai or Mumbai
OnsiteFull Time
Accenture
AccentureNYSE: ACN: Global provider of management consulting and technology services.
2+ YOEMinimum 2 years experience in AWS Analytics, data pipeline and ETL development, data warehousing/data lakes, Python or SQL proficiency, and familiarity with data quality and governance.
AWS Analytics, Python, SQL
Job PostingView all
1mo
Save
Mark Applied
Hide
Lead Data Engineer - Databricks
Hyderabad or Indore or Ahmedabad
OnsiteFull Time
Kanerika
Kanerika: Provides AI, data analytics, and digital transformation consulting services.
5+ YOE5+ years data engineering experience (5–9 preferred) with 2+ years Databricks in production; strong PySpark/Spark SQL, Delta Lake, Unity Catalog, Databricks Workflows, SQL, cloud (Azure preferred), Python, CI/CD (Git), debugging, and client-facing skills.
Databricks, PySpark, Spark SQL, Auto Loader, structured streaming, Delta Lake, Unity Catalog, Databricks Workflows, Lakeflow Declarative Pipelines, Delta Live Tables, Lakeflow Connect, MLflow, Databricks SQL, AI/BI Genie, Airflow, Fivetran, Kafka, Azure Data Factory, Git, Python
Job PostingView all
2mo
Save
Mark Applied
Hide
Data Engineer
Ahmedabad or Dallas
RemoteFull Time
Solvative
Solvative: Builds user-centric digital products and software for global businesses.
2+ YOE2 years experience; strong SQL; proficiency in Python, Scala or Java; ETL/ELT tools (Airflow, dbt, Dagster, Luigi); Spark/Flink (Hadoop legacy); cloud data services (AWS/GCP/Azure, Redshift, BigQuery, Snowflake, Databricks); Git, Terraform, Docker, Kubernetes.
SQL, Python, Scala, Java, Airflow, dbt, Dagster, Luigi, Spark, Flink, Hadoop, AWS, GCP, Azure, Redshift, BigQuery, Snowflake, Databricks, Parquet, Avro, ORC, Git, Terraform, Docker, Kubernetes
Job PostingView all
2mo
Save
Mark Applied
Hide
Senior Data Engineer
Ahmedabad, Gujarat, India
OnsiteFull Time
Creole Studios
Creole Studios: Provides custom software, web, and mobile app development services.
4+ YOE4+ years data engineering experience; strong Python, PySpark, SQL; Spark production experience; cloud (AWS/Azure/GCP); ETL/ELT, data lake/warehouse/lakehouse knowledge; Airflow/Azure Data Factory; Terraform, CI/CD, Git; mandatory industry data-engineering certification.
Python, PySpark, SQL, Spark, AWS, Azure, GCP, Terraform, Airflow, Azure Data Factory, Git, Databricks, Snowflake, CI/CD
Job PostingView all
3mo
Save
Mark Applied
Hide
Data Engineer
Ahmedabad, Gujarat, India
OnsiteFull Time
Decklar
Decklar: AI-powered SaaS platform providing real-time supply chain visibility.
4+ YOEBachelor's or Master's in CS/Data Sciences, 4+ years in data engineering, strong SQL, ETL/ELT experience with Spark/Flink/pandas, cloud (AWS/Azure/GCP), Neo4J/Snowflake/S3 experience, and familiarity with LLM tools.
MySQL, Postgres, Snowflake, S3, Neo4J, Iceberg, pandas, Flink, Spark, Java, Python, HDFS, Cursor, Claude, AWS, Azure, GCP
Job PostingView all