Python__Data Engineering at Cognizant

Python__Data Engineering

United States

OnsiteFull Time

Responsibilities:building pipelines, transforming data, monitoring pipelines

Requirements Summary:Build and maintain ETL/ELT pipelines in Python; use Pandas, PySpark, Dask; work with SQL and NoSQL databases; design data warehouses (Snowflake, Redshift, BigQuery); use Airflow and cloud platforms (AWS, Azure, GCP); ensure data quality and monitoring.

Technical Tools Mentioned:Python, Pandas, PySpark, Dask, SQL, PostgreSQL, MySQL, SQL Server, NoSQL, MongoDB, Cassandra, Snowflake, Redshift, BigQuery, Apache Spark, Hadoop, Apache Airflow, AWS, S3, Glue, Lambda, EMR, Microsoft Azure, Microsoft Data Factory, Microsoft Synapse, GCP, Dataflow

Job Description

Data Pipeline Development
- Build and maintain ETL/ELT pipelines using Python
- Ingest data from multiple sources (APIs, databases, files, streaming systems)
- Optimize pipelines for performance and scalability
- Clean, transform, and validate raw datasets
- Handle structured and unstructured data
  
  Use frameworks like:
  - Pandas
  - PySpark
  - Dask
  - Database & Data Warehousing
  - SQL (PostgreSQL, MySQL, SQL Server)
  - NoSQL (MongoDB, Cassandra)
  - Design schemas and optimize queries
  - Build data warehouses using:
    Snowflake
    Redshift
    BigQuery
    Big Data Technologies
    
    Apache Spark
    
    Hadoop
    
    Process large-scale datasets efficiently
    
    Workflow Orchestration
    
    Apache Airflow
    
    Cloud Platforms
    
    Work on cloud environments:
    
    AWS (S3, Glue, Lambda, EMR)
    
    Azure (Data Factory, Synapse)
    
    GCP (Dataflow, BigQuery)
    
    Data Quality & Monitoring
    
    Implement data validation checks
    
    Monitor pipeline failures and fix bugs
    
    Ensure data reliability and integrity

Python__Data Engineering

Job Description

Data Pipeline Development