HCLTech
Posted 2w ago

Apache Spark Developer - Scala, Python (142051)

HCLTech
United States
OnsiteFull Time
Responsibilities
  • developing applications
  • implementing pipelines
  • optimizing performance
Requirements
  • Develop and implement data processing applications using Apache Spark, Scala, and Python
  • Build ETL pipelines
  • Troubleshoot and optimize large-scale workflows
Technical tools mentioned
Apache SparkScalaPython

Job description

Requisition ID 142051 - Posted 


Job Summary

The Developer will be responsible for developing and implementing applications using apache spark, scala, and python. The primary focus will be on coding, testing, and debugging software programs for complex data processing and analytics tasks.

Key Responsibilities

1. Develop efficient code using apache spark, scala, and python for data processing applications.

2. Collaborate within team to gather and understand requirements for software development projects.

3. Implement data pipelines and etl processes to extract, transform, and load data from various sources.

4. Optimize and tune application performance for largescale data processing workflows.

5. Troubleshoot and debug issues to ensure the software meets quality and performance standards.

6. Stay updated on emerging technologies and best practices in data engineering and analytics.

Skill Requirements

1. Knowledge of apache spark for distributed data processing.

2. Programming skills in scala for developing scalable applications.

3. In-depth knowledge of python programming language for data manipulation and analytics.

4. Familiarity with working with big data technologies and frameworks.

5. Understanding of data structures, algorithms, and software design principles.

6. Good analytical and problem-solving abilities.

7. Good communication and teamwork skills.

Other Requirements

1.Cloudera Certified Developer for Apache Spark (CCD) certification is a plus.


About HCLTech

Global provider of information technology services and software consulting.

Similar jobs

Apache Spark Developer roles
3mo
Save
Mark Applied
Hide
Apache Spark Developer
Herndon or Springfield or St. Louis or Aurora
$150k-$225k/yr OnsiteFull Time
Absolute Business Solutions Corp
Absolute Business Solutions Corp: Provides technology and intelligence services to the US government.
5+ YOETS/SCI eligibility preferred (min Secret), bachelor\u0002s degree plus 5+ years data engineering/Spark experience; strong Spark, PySpark, Python, and large-scale data processing skills; Kubernetes and container experience; performance tuning and scripting skills.
Apache Spark, PySpark, Scala, Delta Lake, Parquet, Structured Streaming, Kubernetes, Docker, S3, AWS, GCP, Azure, Git, Jenkins, Python, Bash, Terraform, Iceberg