Data engineer with Java
Responsibilities:
Data Engineer Job Requirements
- Data Pipeline Expertise - Capable of implementing a diverse range of data solutions with significant autonomy, demonstrating the ability to independently build and maintain complex ETL (Extract, Transform, Load) pipelines
- Possesses advanced skills in pipeline complexity management and orchestration, with a proven track record of developing high-performance data products accessible via APIs for critical tier 1 trading applications
- Demonstrates expertise in archiving data products to data lakes for comprehensive analytics purposes
- Data Quality and Advanced Analytics - Robust proficiency in Data Quality, Data Validation, and Anomaly Detection techniques - Experience with AI/ML and Large Language Model (LLM) approaches to data analysis and validation - Ability to implement sophisticated data integrity and verification processes
Qualifications
Technical Skill Requirements:
- Programming and Cloud Technologies - Expert-level skills in: - Python - SQL - AWS Cloud Services: - Cloud Storage - Lambda - ECS - Glue Data Quality - HDMI - Redshift Machine Learning - OpenSearch - RDS
- API and Resilience Technologies - Advanced skills in building resilient API architectures, including: - Microservices design patterns - API gateway configuration - Circuit breaker implementations - Fault tolerance and error handling mechanisms - Expertise in API development and management tools: - Swagger/OpenAPI - Postman - GraphQL - RESTful API design principles
- Proficiency in resilience engineering techniques: - Distributed system design - Load balancing - Failover strategies - Horizontal and vertical scaling approaches - Containerization and orchestration: - Docker - Kubernetes - Helm charts
- Monitoring and observability: - Prometheus - Grafana - ELK Stack (Elasticsearch, Logstash, Kibana) - Distributed tracing with Jaeger or Zipkin Additional Technical Competencies
- Proficient in Apache Flink - Advanced Java programming skills - Proven ability to create and optimize complex data products - Demonstrated Subject Matter Expertise (SME) in data engineering principles ## Professional Attributes
- Capability to work independently through complex data challenges - Robust preference for candidates with prior experience in investment data domain and related technologies
- Key Differentiators - Deep understanding of data engineering best practices - Ability to design scalable, efficient, and resilient data solutions - Proven track record of solving intricate data infrastructure problems with robust API architectures
Benefits at IntelliBee
- Long-Term Stability: Join us on a multi-year opportunities with room to grow.
- Comprehensive Health Coverage: Access quality healthcare benefits to keep you and your family well.
- Future Planning: Enroll in our 401(k) program and invest in your financial security.
- GC Assistance: We support immediate Green Card processing, if required.
