Data Engineer

Benefits

Additional Information

Job Description: Job Description Details Business Title Data Engineer Years of Experience Min 3 and max upto 7. Job Descreption Looking for a hands‑on Senior Data Engineer - Azure & Databricks with up to 6 years of total experience to build, optimize, and maintain scalable cloud data platforms. This is an individual contributor role, focused on developing reliable data pipelines and analytics‑ready datasets using Microsoft Azure and Databricks. The role requires strong hands‑on expertise in SQL and Python, along with experience building batch (and basic streaming) data pipelines in a cloud environment. You will collaborate closely with architects, analytics teams, QA, DevOps, and business stakeholders in a global delivery model. Must have skillsHands‑on experience delivering Azure‑based data engineering solutions Cloud & Data Engineering (Azure) Strong hands‑on experience with Azure data services, including: Azure Data Lake Storage Gen2 (ADLS) Azure Data Factory (ADF) Azure Databricks Azure Synapse Analytics Experience designing cloud‑native data lakes and analytical data stores Solid understanding of batch data pipelines and basic streaming concepts SQL & Python (Mandatory) Strong SQL skills (mandatory) Writing complex queries, joins, aggregations, and transformations Hands‑on experience working with large datasets in Synapse / Databricks Strong Python skills (mandatory) Python for ETL / ELT and data engineering use cases Hands‑on experience with PySpark in Databricks Strong understanding of data modeling, transformations, and query performance tuning Data Processing & Engineering Hands‑on experience with Spark / PySpark on Databricks Experience handling structured and semi‑structured data Understanding of partitioning, schema evolution, and data validation concepts DevOps & Platform Basics Working knowledge of Infrastructure as Code (ARM Templates and/or Terraform) Basic experience with CI/CD pipelines (Azure DevOps or GitHub Actions) Understanding of logging and monitoring using Azure Monitor / Log Analytics Collaboration Ability to work effectively with architects, QA, DevOps, and business stakeholders Good communication skills for explaining technical concepts clearly Good to have skillsExposure to streaming technologies such as Azure Event Hubs or Kafka Familiarity with Lakehouse architecture and Delta Lake concepts Experience integrating Azure data platforms with BI tools (Power BI preferred) Basic knowledge of data governance and data quality frameworks Awareness of Azure cost‑optimization best practices Experience working in Agile delivery models, especially with global clients Key responsibiltesData Engineering & Development Build scalable ETL / ELT pipelines using Azure and Databricks Develop SQL‑based transformations and Python / PySpark pipelines Ingest and process data using ADF, Databricks, and ADLS Build analytics‑ready data models optimized for performance and cost Platform & Operations Support deployment and execution of data pipelines across environments Monitor data pipeline health, performance, and data quality Troubleshoot pipeline failures and perform root‑cause analysis Follow Azure security, reliability, and scalability best practices Collaboration & Delivery Work closely with architects and product teams to understand requirements Translate business and analytics needs into working Azure data solutions Contribute to documentation, code reviews, and engineering standards Education Qulification 1. Bachelor's or Master Degree or equivalent Degree Certification If Any 1. Microsoft Certified: Fabric Data Engineer Associate 2.Microsoft Certified: Azure Solutions Architect Expert 3. Snowflake Core Shift timing 12 PM to 9 PM and / or 2 PM to 11 PM - IST time zone Location: DGS India - Pune - Kharadi EON Free Zone Brand: Merkle Time Type: Full time Contract Type: Permanent