Senior Data Engineer - Azure Databricks

Responsibilities

• Design, develop, and optimize end-to-end data pipelines using Azure Databricks
• Build and maintain scalable ETL/ELT frameworks for structured and unstructured data
• Ingest data from multiple sources including APIs, databases, files, and streaming systems
• Work with Azure Data Lake Storage (ADLS Gen2) for efficient data storage and retrieval
• Develop data transformation logic using PySpark and Spark SQL
• Ensure data quality, reliability, and performance across pipelines
• Implement data modeling concepts to support analytics and reporting use cases
• Collaborate with Data Scientists, BI teams, and business stakeholders to understand data requirements
• Monitor, troubleshoot, and optimize Databricks jobs for performance and cost efficiency
• Apply best practices around security, governance, and compliance within Azure
• Participate in code reviews, documentation, and continuous improvement initiatives
• 5+ years of hands-on experience as a Data Engineer
• Strong experience with Azure Databricks
• Proficiency in PySpark , Spark SQL , and Python
• Experience working with Azure Data Lake Storage (ADLS Gen2)
• Solid understanding of data warehousing and data modeling concepts
• Experience building batch and/or streaming data pipelines
• Familiarity with Azure Data Factory or similar orchestration tools
• Strong SQL skills and experience with relational databases
• Experience using Git or similar version control systems
• Strong analytical, problem-solving, and communication skills
• Experience with Delta Lake
• Knowledge of streaming technologies (Kafka, Azure Event Hubs, Spark Structured Streaming)
• Exposure to CI/CD pipelines using Azure DevOps
• Experience working in Agile/Scrum environments
• Understanding of cloud cost optimization strategies

Additional Information

We are looking for a skilled Data Engineer with 5+ years of experience to design, build, and maintain scalable data pipelines and platforms using Azure and Databricks . The ideal candidate will work closely with analytics, data science, and business teams to deliver reliable, high-performance data solutions that enable data-driven decision-making.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Blend360? Share your experience