Data Engineer

About the role

🎯 Key Responsibilities 🔹 Data Engineering & Integration Design, build, and optimize ETL/ELT pipelines using Apache Spark, PySpark, Databricks, Azure Synapse , or equivalent platforms. Develop scalable batch and real-time data processing solutions. Integrate data from Core Banking, Payments, Treasury, Trade Finance, CRM, Compliance, and Risk systems. Develop and maintain enterprise data models including 3NF, Dimensional Modeling, and Data Vault 2.0 . 🔹 Streaming & Modern Data Platforms Build and operationalize real-time streaming pipelines using Kafka, Confluent, or Azure Event Hubs . Support data platform modernization initiatives, including migration from legacy platforms (e.g., Teradata, DB2) to cloud-native environments such as Snowflake, Databricks, or Azure Synapse . Implement scalable cloud-based data lake and data warehouse architectures. 🔹 Data Quality & Governance Implement data quality, validation, lineage, and observability frameworks using tools such as Great Expectations, Deequ, or dbt . Collaborate with Governance and Security teams to ensure compliance with enterprise data standards. Support metadata management, cataloging, and lineage initiatives using Azure Purview, Apache Atlas, or Collibra . 🔹 Regulatory & Compliance Support Support regulatory reporting and risk data flows including:MAS 610MAS 649Basel III / Basel IVIFRS 9 / IFRS 17BCBS 239 Ensure data security controls including encryption, tokenization, masking, RBAC, and audit logging are implemented. 🔹 DevOps & MLOps Develop CI/CD pipelines using Azure DevOps, GitHub Actions, or Terraform . Collaborate with Data Scientists and AI teams to deploy ML feature stores and model-serving pipelines. Support automation and Infrastructure-as-Code (IaC) initiatives. 🛠 Required Technical Skills Programming Languages Python PySpark SQL Scala Data Platforms Azure Data Lake Azure Synapse Analytics Databricks Snowflake Data Orchestration Apache Airflow Azure Data Factory (ADF) dbt Streaming Technologies Apache Kafka Confluent Platform Azure Event Hubs Data Governance Azure Purview Apache Atlas Collibra Security & Compliance Encryption Tokenization Role-Based Access Control (RBAC) Audit Logging DevOps & Infrastructure Terraform Azure DevOps GitHub Actions 🎓 Qualifications Bachelor's or Master's Degree in Computer Science, Data Engineering, Information Technology, or related discipline. 6-10 years of Data Engineering experience. Minimum 3 years of experience within Banking, Financial Services, Insurance, or Capital Markets environments. Strong experience designing and implementing cloud-based data platforms on Azure and/or AWS. Hands-on experience with batch and real-time data processing frameworks. Understanding of regulatory reporting and risk data management frameworks.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at KNOWLEDGESG GLOBAL PTE. LTD.? Share your experience