Data Lead (Databricks & Power BI) - Offshore

About the role

Responsibilities API Integration & Data Extraction Lead the integration of Fenergo APIs to extract relevant KYC and AML data, ensuring seamless connectivity and data flow between systems Design, develop, and maintain scalable data pipelines and ETL processes to support data ingestion from various sources, including databases, APIs, and flat files Ensure robust data extraction processes that maintain data quality and compliance with regulatory requirements Data Processing & Pipeline Development Utilize Databricks and Apache Spark to design and implement robust data processing pipelines, ensuring high data quality and performance Work with DataFrames for transforming data and implementing the Medallion Architecture Execute SQL queries for data extraction, manipulation, and complex data operations Join datasets and add fields to reports to provide comprehensive analytical insights Leverage AI tools in Databricks to assist with data workflows and optimization Use notebooks as data transformation pipelines for efficient data processing Data Analysis & Interpretation Analyze and interpret complex data sets to identify trends, patterns, and anomalies that can inform business decisions related to client and investor lifecycle management Understand and navigate the data model to ensure accurate data representation and reporting Conduct regular data quality assessments and audits to ensure data integrity and compliance with industry standards Perform root cause analysis to swiftly identify data issues and collaborate with relevant teams to implement effective solutions Data Visualisation & Reporting Architect and develop interactive dashboards and reports in Power BI, translating complex data into clear, actionable insights for clients, leadership, and stakeholders Develop and maintain dashboards and reports to provide insights into key performance indicators (KPIs) and operational metrics for KYC/AML processes Create visual representations that highlight critical data points for regular reporting to clients and senior management Ensure reports meet the needs of both technical and non-technical stakeholders Collaboration & Stakeholder Management Collaborate with cross-functional teams, including IT, Compliance, Risk Management, business analysts, and senior management, to gather data requirements and deliver strategic insights Engage with clients and internal stakeholders to understand their reporting needs and ensure alignment with business objectives Work closely with KYC/AML operations teams to ensure data solutions support compliance and regulatory requirements Act as a bridge between technical teams and business users, translating complex data concepts into actionable business insights Documentation & Governance Maintain comprehensive documentation of data processes, API integrations, data flows, data management processes, and reporting solutions for future reference and compliance Document data governance practices and ensure adherence to data quality best practices Ensure all data handling complies with regulatory standards and internal policies Continuous Improvement & Problem-Solving Recommend long-term product solutions to enhance data quality, accessibility, and usability Identify opportunities for process optimization and automation in data workflows Stay up-to-date with industry trends and best practices in data engineering, analysis, and management Proactively identify and resolve data-related issues, ensuring timely and accurate reporting Demonstrate creativity and insightfulness in developing dynamic approaches to complex data challenges Quality Assurance Ensure data integrity throughout all pipelines and reporting mechanisms Implement data validation and quality control measures Monitor data processes and implement control mechanisms to ensure reliability Skills Core Data Engineering Skills (Required): Proficiency in Databricks and Apache Spark for data processing and pipeline development Strong knowledge of Power BI for data visualization and reporting, with ability to create executive-level dashboards Expert-level proficiency in SQL for data querying, manipulation, and complex analytical operations Experience with programming languages such as Python or R for data analysis and automation Strong understanding of data warehousing concepts and ETL processes Knowledge of data modeling concepts and best practices for data management Understanding of the Medallion Architecture and data lakehouse principles Experience working with DataFrames for data transformation Ability to leverage AI tools in Databricks to optimize data workflows API & Integration Skills (Required): Strong experience in API integration for data extraction and system connectivity Ability to ensure seamless data flow between multiple systems Cloud & Infrastructure (Required): Experience with cloud platforms, particularly Azure or AWS Knowledge of Git connection to Databricks for version control Experi

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Photon? Share your experience