Data Engineer

About the role

ESSENTIAL ROLES & RESPONSIBILITIES Identify and understand customer data-centric use cases within regulated financial services environments Design and implement data ingestion, processing, and transformation pipelines on Azure Build and maintain data pipelines for cleaning, normalisation, enrichment, and preparation Apply appropriate data modelling techniques and architecture patterns, with a strong focus on medallion architecture Orchestrate, monitor, and optimise Azure Databricks jobs and Azure Data Factory pipelines across development, UAT, and production environments Configure platforms, clusters, and compute resources to optimise performance, cost, and reliability Use automated CI/CD pipelines to manage, deploy, and version data artefacts and pipelines Operationalise workflows developed by analysts and data scientists Support customers in adopting Azure data, analytics, and machine learning services Ensure secure storage, processing, and quality of customer data Ensure networking and security best practices are applied when designing and operating data solutions Design solutions for processing large volumes of data using batch and streaming approaches Collaborate with analytics teams on data visualisation best practices and reporting enablement Ensure all solutions are well-documented, including pipelines, schemas, transformations, and operational runbooks GOVERNANCE & REPORTING Maintain accurate documentation of data pipelines, schemas, transformations, and deployment processes Support data governance initiatives including lineage, metadata management, and access control Contribute to service reporting, risk tracking, and continuous improvement actions Ensure data environments are audit-ready and aligned with governance standards TECHNOLOGY STACK (AZURE) Cloud Platform: Microsoft Azure Data Engineering & Analytics: Azure Databricks (development, UAT, and production) Azure Data Factory Azure Synapse Analytics (where applicable) Machine Learning & AI : Azure Machine Learning (limited non-production usage) Azure Document Intelligence Databases: Microsoft SQL Server / Azure SQL Database (primary platforms) PostgreSQL (limited use) MySQL (limited use) Data Processing: Batch and streaming data pipelines Security & Governance: Role-based access control (RBAC) Data encryption and key management Audit logging and monitoring DevOps: CI/CD pipelines for data artefacts and infrastructure BEHAVIOURAL COMPETENCIES - ORGANISATIONAL & BEHAVIOURAL FIT Positive mindset and enthusiasm for learning new technologies Collaborative and supportive team player Strong sense of ownership and accountability Methodical, analytical approach to problem-solving Strong understanding of ethical data usage in regulated environments CRITICAL COMPETENCIES - TECHNICAL FIT Essential: Strong SQL skills Programming experience with Python and/or Scala Hands-on experience with Azure-based data platforms Experience designing, building, and maintaining data pipelines Strong understanding of data modelling (relational and analytical), including medallion architecture Experience orchestrating and optimising Databricks and Data Factory workloads Experience using CI/CD pipelines for data and analytics solutions Strong awareness of security, networking best practices, GDPR, and PII handling Desirable: Experience with Azure Databricks in production environments Familiarity with Azure Machine Learning and AI services Exposure to data visualisation tools (e.g. Power BI) Experience with big data frameworks (Spark, Kafka) Knowledge of data governance, lineage, and metadata tooling SHIFT & WORKING PATTERN Standard business hours, with participation in an on-call rota as required Occasional weekend engineering coverage will be required, typically limited to a small number of planned weekends per year to support business continuity, resilience testing, or disaster recovery activities

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Claranet Limited? Share your experience