We are looking for seasoned Data Engineer to work with our team and our clients to develop enterprise grade data platforms, services, and pipelines. We are looking for more than just a "Data Engineer", but a technologist with excellent communication and customer service skills and a passion for data and problem solving.
- Lead and architect data pipelines and ingest patterns to move raw data from data producers to an enterprise data ecosystem, with a focus on performance and reliability
- Assess and understand the ETL jobs, workflows, BI tools, and reports
- Address technical inquiries concerning customization, integration, enterprise architecture and general feature / functionality of data products
- Experience in crafting database / data warehouse solutions in cloud (Preferably AWS. Alternatively Azure, GCP).
- Key must have skill sets – Python, SQL, Databricks, AWS Data Services
- Support an Agile software development lifecycle
- You will contribute to the growth of our Data Exploitation Practice!
- Experience supporting ICE contract work.
- Experience working in Design Intelligence.
- Bachelor’s Degree and 6+ years of total experience or equivalent experience and education.
- 6+ years direct experience in Data Engineering with experience in tools such as:
- Big data tools: Hadoop, Spark, Kafka, etc.
- Relational SQL and NoSQL databases, including Postgres and Cassandra.
- Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- AWS cloud services: EC2, S3, RDS, Glue, Step Functions, Lamda, EMR, DynamoDB, DocumentDB, Redshift, Aurora, Athena
- Data Platforms: Databricks, Snowflake
- Data streaming systems: Kafka, Storm, Spark-Streaming, etc.
- Languages: Python, R, Scala, Go
- Ability to inspect existing data pipelines, discern their purpose and functionality, and re-implement them efficiently in Databricks.
- Advanced working SQL knowledge and experience working with relational databases
- Advanced working knowledge and of NoSQL databases
- Experience with message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Experience manipulating, processing, and extracting value from large, disconnected datasets.
- Experience manipulating structured and unstructured data for analysis.
- Experience with data modeling tools and processes.
- Experience aggregating and transforming data from multiple datasets to create data products
- Experience working in an Agile environment
