Data Engineer

About the role

We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data ingestion and transformation pipelines. The ideal candidate will be responsible for developing end-to-end data pipelines, implementing data quality frameworks, and enabling reliable data delivery into a governed data lake environment. Responsibility Design and implement ETL transformations based on defined data mapping specifications. Build and maintain data quality validation frameworks to ensure completeness, schema conformance, and referential integrity. Configure error handling, exception management, and dead-letter queue patterns for failed ingestion records. Develop and maintain Infrastructure as Code (IaC) using Terraform or AWS CDK for data platform resources. Execute unit testing and integration testing to validate pipeline functionality and data accuracy. Support User Acceptance Testing (UAT) by validating output datasets, schemas, and record counts with business stakeholders. Create and maintain technical documentation including pipeline configurations, runbooks, and data flow diagrams. Participate in Agile ceremonies including sprint planning, stand-ups, demos, and code reviews. Requirement 5+ years of experience in Data Engineering or Data Platform Development. Strong hands-on experience with AWS Glue, AWS Lake Formation, Amazon S3, and Amazon Athena. Proficiency in Python, PySpark, and SQL. Experience building ETL/ELT pipelines in cloud-native environments. Hands-on experience with Infrastructure as Code tools such as Terraform or AWS CDK. Strong understanding of data modeling, data governance, and metadata management. Experience with automated testing and deployment practices.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at PROJECT SEARCH PTE. LTD.? Share your experience