HiringCafe
📋 External Recruiting Agencies

TalPro is a recruitment and staffing agency that sources candidates for other companies, rather than acting as a direct employer for the roles advertised.

This company was flagged and excluded from default search results. Proceed with caution.

Save Job
Posted 7mo ago

Data Software Engineer – Spark, Python, Databricks (L2 / L4)

@ Talpro India
View All Jobs
Bengaluru, Karnataka, India
₹21-₹42/yrHybridFull Time
Responsibilities:design systems, develop apps, build pipelines
Requirements Summary:Data Software Engineer with Spark, Python, and Databricks; L2 (4–5 yrs) or L4 (8–12 yrs); hybrid in Bangalore; FTE.
Technical Tools Mentioned:Apache Spark, Python, Hadoop, Hive, Impala, Kafka, RabbitMQ, NoSQL, HBase, Cassandra, MongoDB, AWS Databricks, Azure Databricks
Save
Mark Applied
Hide Job
Report & Hide
Job Description


Data Software Engineer – Spark, Python, Databricks (L2 / L4)

Experience:

  • L2: 4–5 Years

  • L4: 8–12 Years

Mode: FTE (Full-Time Employment)

Job Location: Bangalore

Work Mode: Hybrid

Notice Period:

  • L2: Immediate to 15 days

  • L4: Immediate to 15 days  

Drive Type: F2F

Drive Location: Bangalore

CTC Band:

  • L2: Up to 21 LPA

  • L4: Up to 42 LPA


Role Overview

We are hiring Data Software Engineers with strong expertise in Apache Spark, Python, and AWS/Azure Databricks. The ideal candidates will have deep Big Data engineering experience, strong distributed systems knowledge, and the ability to work on complex end-to-end data platforms at scale.


Key Responsibilities

Big Data Engineering

  • Design and build distributed data processing systems using Spark and Hadoop.

  • Develop and optimize Spark applications, ensuring performance and scalability.

  • Create and manage ETL/ELT pipelines for large-scale data ingestion and transformation.

Streaming & Event Processing

  • Build and manage real-time streaming systems using Spark Streaming or Storm.

  • Work with Kafka / RabbitMQ for event-driven ingestion and messaging patterns.

Cloud & Databricks Engineering

  • Develop & optimize workloads on AWS Databricks or Azure Databricks.

  • Perform cluster management, job scheduling, performance tuning, and automation.

Data Integration & Storage

  • Integrate data from diverse sources: RDBMS (Oracle, SQL Server), ERP, file systems.

  • Work with query engines like Hive and Impala.

  • Experience with NoSQL stores: HBase, Cassandra, MongoDB.

Programming & Scripting

  • Strong hands-on coding in Python for data transformations and automations.

  • Strong SQL skills for data validation, tuning, and complex queries

Team Leadership (L4)

  • Provide technical leadership and mentoring to junior engineers.

  • Drive solution design for Big Data platforms end-to-end

Ways of Working

  • Work in Agile teams, participate in sprint ceremonies and planning.

  • Collaborate with engineering, data science, and product teams.


Required Skills & Expertise (Both L2 & L4)

  • Apache Spark – Expert level (core, SQL, streaming)

  • Python – Strong hands-on

  • Distributed computing fundamentals

  • Hadoop ecosystem: Hadoop v2, MapReduce, HDFS, Sqoop

  • Streaming systems: Spark Streaming / Storm

  • Messaging: Kafka or RabbitMQ

  • SQL – Advanced (joins, stored procedures, query optimization)

  • NoSQL: HBase, Cassandra, MongoDB

  • ETL frameworks & data pipeline design

  • Hive / Impala querying

  • Performance tuning of Spark jobs

  • AWS or Azure Databricks

  • Experience working in Agile


Experience & Level Mapping


L2 – Mid-Level (4–5 Yrs)

  • Skills: Spark, Python, AWS

  • Notice Period: Immediate – 20 Days

  • CTC Band: Up to 21 LPA

L4 – Senior-Level (8–12 Yrs)

  • Skills: Spark, Python, Azure Databricks

  • Notice Period: 15 Days (Nov joiners) OR Jan joiners

  • CTC Band: Up to 42 LPA