We are looking for an experienced Senior Python Data Engineer with strong expertise in Python development, microservices architecture, data engineering, and backend API development. The ideal candidate will have hands-on experience building scalable Python-based applications and data services, designing robust REST APIs, performing complex data transformations, and developing high-performance backend frameworks to support enterprise data-driven applications.
This role requires strong proficiency in Python web frameworks, asynchronous programming, microservices architecture, ETL processes, database management, and modern software engineering best practices. Exposure to big data technologies, DevOps practices, and cloud-native architectures will be highly advantageous.
Requirements
Key Responsibilities
- Design, develop, and maintain scalable Python-based web frameworks and backend services.
- Develop robust RESTful APIs using Python frameworks such as FastAPI, Flask, or Django.
- Build and maintain microservices-based applications supporting enterprise data platforms.
- Design and implement data transformation pipelines for data ingestion, processing, and enrichment.
- Develop backend systems to efficiently serve datasets through APIs and other interfaces.
- Collaborate with cross-functional teams to deliver scalable and secure data-driven applications.
- Design and maintain relational and NoSQL database schemas.
- Develop efficient ETL (Extract, Transform, Load) processes for enterprise data requirements.
- Optimize application and database performance to ensure scalability and reliability.
- Implement asynchronous programming techniques using asyncio and optimize concurrent connections and I/O operations.
- Ensure data integrity, consistency, and quality across data transformation pipelines.
- Participate in code reviews and contribute to software engineering best practices.
- Troubleshoot, debug, and resolve production issues across applications and data pipelines.
- Collaborate closely with stakeholders and engineering teams throughout the SDLC.
Required Technical Skills
Core Python Development
- Strong hands-on expertise in Core Python.
- Experience building enterprise-grade Python applications.
- Strong understanding of:
- Python programming concepts
- Object-oriented programming
- Asynchronous programming (asyncio)
- Concurrent connections handling
- I/O optimization techniques
- Clean code and software design principles
Python Web Frameworks
Hands-on experience with one or more of the following:
- FastAPI
- Flask
- Django
- Pyramid
Strong understanding of:
- RESTful API development
- Routing
- Authentication mechanisms
- Testing frameworks
- Database integrations
- Framework architecture and scalability
Microservices Architecture
Strong experience with:
- Microservices-based application design
- Service-oriented architectures
- API integrations
- Distributed systems
- Enterprise backend services
Knowledge of:
- API Gateways
- Service communication patterns
- Scalable backend architectures
Security & API Management
Experience implementing:
- OAuth
- JWT Authentication
- API Security Best Practices
- Authorization mechanisms
- Encryption standards
- Secure API development practices
Data Engineering & Data Transformations
Strong hands-on experience with:
- Data transformation pipelines
- ETL processes
- Data cleansing and enrichment
- Data aggregation techniques
- Data quality and integrity management
Experience in:
- Dataset preparation
- Data processing workflows
- Enterprise data platforms
Database Management
Experience working with:
Relational Databases
- PostgreSQL
- MySQL
- SQL Server (preferred)
NoSQL Databases
- MongoDB
- Redis (good to have)
Strong knowledge of:
- Database schema design
- Performance optimization
- Transactions management
- Efficient data retrieval techniques
ORM Frameworks
Hands-on experience with:
- SQLAlchemy
- Django ORM
Ability to perform:
- CRUD operations
- Query optimization
- Transaction handling
- Data modelling
DevOps & CI/CD
Experience with:
- CI/CD pipeline development and maintenance
- Git-based workflows
- Version control best practices
- Branching strategies
- Code reviews
- Production deployments
Big Data Technologies (Preferred)
Exposure to:
- Apache Spark
- Hadoop
- Apache Kafka
Understanding of:
- Data warehousing concepts
- Large-scale data processing architectures
Scripting Skills
Strong proficiency in:
- SQL
- Shell Scripting
- Python scripting
Cloud & Containerization (Good to Have)
Exposure to:
- AWS
- Microsoft Azure
- Google Cloud Platform (GCP)
- Docker
- Kubernetes
Data Governance (Good to Have)
Knowledge of:
- Data governance principles
- Data security best practices
- Compliance frameworks
- Enterprise data management practices
Preferred Additional Skills
- Experience with R Programming Language.
- Experience working on enterprise-scale data engineering initiatives.
- Exposure to cloud-native architectures and containerized deployments.
Required Competencies
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management capabilities.
- Strong debugging and troubleshooting skills.
- Ability to work effectively in Agile and collaborative environments.
- Strong ownership mindset and attention to detail.
- Ability to adapt quickly to evolving business and technical requirements.
- Strong documentation and technical communication skills.
