Centre for Strategic Infocomm Technologies
Posted 1y ago

Senior Data Engineer (Big Data Processing)

Centre for Strategic Infocomm Technologies
Singapore, Singapore, Singapore
OnsiteFull Time
Responsibilities
  • Design pipelines
  • Collaborate stakeholders
  • Optimize pipelines
Requirements
  • Senior data engineer with 5+ years in data pipelines
  • Proficient in Scala/Java
  • Experience with Spark, Kafka, Hadoop
  • ETL coding
  • Data modeling
Technical tools mentioned
SparkKafkaHadoopElasticKuduScalaJavaETL

Job description

CSIT develops products to advance the national security interests of Singapore. We use our products in a wide range of use-cases, including but not limited to Counter-terrorism and Computer Network Defence. We are looking for talented senior data engineers to lead and build mission-critical products that identify, analyse and investigate threats.


CSIT develops products to advance the national security interests of Singapore. We use our products in a wide range of use-cases, including but not limited to Counter-terrorism and Computer Network Defence. We are looking for talented senior data engineers to lead and build mission-critical products that identify, analyse and investigate threats.


Role
  • As a senior data engineer in a team that is responsible for mission-critical data analytics systems, you will contribute to the development and maintenance of high-performance data pipelines and systems related to national security. You will be involved in complex technical projects to collect, parse and process large datasets with the aim to turn them into useful information and insights. Join us to seize the opportunity to use cutting-edge technologies to protect the nation.


  • Responsibilities
  • Design and build data pipelines that are scalable, robust and maintainable to ingest crucial data using cutting-edge platforms and technologies
  • Work with product managers, engineering managers and key stakeholders to deliver impactful data products that meet our business needs
  • Optimise performance of large-scale data pipelines for low-latency, high-throughput and reliability
  • Ensure data integrity, quality and governance through robust design and operational monitoring
  • Troubleshoot and resolve complex issues in critical data pipelines
  • Implement best practices in data engineer, including testing, orchestration and monitoring of data pipelines
  • Lead technical discussions with the team and key stakeholders to shape the future of our data products
  • Mentor junior data engineers and contribute to building a strong data engineering culture within the team


  • Requirements
  • Bachelor's degree in computer science or equivalent
  • 5+ years of experience in data engineering, with a strong focus on building large-scale data pipelines
  • Proven experience in writing optimised ETL code to handle large datasets
  • Proficient in programming languages such as Scala or Java
  • Familiarity with the following technologies: Spark, Kafka, Hadoop, Elastic stack, Kudu
  • Strong understanding of modern data architectures and data modelling paradigms
  • Able to communicate technical details effectively and clearly
  • Good interpersonal, analytical and problem-solving sklils to get things done in a dynamic environment


  • Why join us?
  • The work is purposeful and meaningful
  • You will get to work with talented and driven people
  • We work with modern technologies and tech stacks
  • We have excellent engineering culture and work-life balance
  • We aspire for engineering and operational excellence
  • We empower to innovate


  • As CSIT is an agency under the Ministry of Defence (Singapore), only Singapore Citizens will be considered.

    About Centre for Strategic Infocomm Technologies

    Develops digital technologies for Singapore's national security missions.

    Year founded
    2003
    Employees
    750
    Organization type
    Government
    Headquarters
    SG

    Similar jobs

    Senior Data Engineer roles near Singapore, Singapore
    5mo
    Save
    Mark Applied
    Hide
    Senior Data Engineer
    Singapore, /, Singapore
    OnsiteFull Time
    Airwallex
    Airwallex: Global financial platform for business payments and money management.
    5+ YOEBachelor’s degree or higher; 5+ years ETL experience; streaming data tech; SQL and data warehousing; GCP, Databricks, Airflow; data governance; strong comms; fintech familiarity preferred.
    Apache Kafka, Apache Flink, SQL, MySQL, PostgreSQL, Oracle, Google Cloud Platform, Databricks, Airflow, Data Governance
    1y
    Save
    Mark Applied
    Hide
    Senior Data Engineer
    Singapore
    OnsiteFull Time
    Razer
    Razer: Sells high-performance gaming hardware, software, and financial services.
    3+ YOE3+ years of experience in data engineering, strong programming skills in Python and SQL, and experience with cloud infrastructure.
    Python, SQL, AWS, Azure, Google Cloud Platform, Terraform, Docker, Kubernetes, Airflow, Prefect, Spark, Dask, Data Build Tool (DBT)
    3mo
    Save
    Mark Applied
    Hide
    Senior Data Warehouse Developer (SG)
    Singapore, /, Singapore
    HybridFull Time
    Singapore University of Social Sciences
    Singapore University of Social Sciences: Provides applied degrees and lifelong education in Singapore.
    5+ YOE5+ years data warehousing; cloud and on-prem; SQL; ETL; BI tools.
    AWS Redshift, Azure Synapse Analytics, Google BigQuery, AWS Glue, Azure Data Factory, IBM Infosphere DataStage, Power BI, MicroStrategy
    2mo
    Save
    Mark Applied
    Hide
    Senior DBA / Data Platform Engineer
    Singapore, Singapore, Singapore
    HybridFull Time
    Sourceability
    Sourceability: Digital distributor of electronic components and supply chain solutions.
    5+ YOE5+ years of DBA/data platform experience; strong MS SQL Server; HA/DR, performance tuning, backup/recovery; ETL/ELT and data pipelines; collaboration with engineering and DevOps.
    Microsoft SQL Server, SQL Server AlwaysOn, replication, log shipping, clustering, SSIS, Azure DevOps, Jira, Git, data pipelines, ETL/ELT, Power BI, Azure Monitor, Datadog, Grafana, Prometheus, Azure SQL, Azure Managed Instance, AWS RDS for SQL Server, PostgreSQL, MySQL, Redis, Elasticsearch, MongoDB, dbt, Airflow, Azure Data Factory, Sentry
    5mo
    Save
    Mark Applied
    Hide
    Senior AI Data Center Network Engineer
    Singapore or Kuala Lumpur or Taipei
    OnsiteFull Time
    Bitdeer
    BitdeerNASDAQ: BTDR: Operates cryptocurrency mining and high-performance computing data centers.
    10+ YOE10+ years in large-scale network operations/architecture; Bachelor’s in CS/Telecommunications or related field; strong TCP/IP, routing (BGP/OSPF/ISIS), DCN technologies; HPC/AI networking experience; Linux proficiency; multiple advanced certifications.
    Linux, TCP/IP, BGP, OSPF, ISIS, EVPN-VXLAN, NVIDIA Quantum switches, NVIDIA Spectrum switches, ConnectX NICs, NCCL, NVIDIA UFM, ROCEv2, Zabbix, Prometheus, Grafana, Telegraf, Python, Go, Ansible, Terraform, InfiniBand, SHARP, DCQCN