Data Engineer

Responsibilities

• Display technical expertise in data analytics focusing on a team of diversified technical competencies.
• Build and maintain accurate and scalable data pipeline and infrastructure such as SQL Warehouse, Data Lakes, etc. using Cloud platforms (e.g.: MS Azure, Databricks).
• Proactively work with business stakeholders to understand data lineage, definitions, and methods of data extraction.
• Write production-grade SQL and PySpark code to create data architecture.
• Consolidate SQL databases from multiple sources, data cleaning, and manipulation in preparation for analytics and machine learning.
• Use data visualization tools such as Power BI to create professional quality dashboards and reports.
• Write good quality documentation for data processing for different projects to ensure reproducibility.
• Living Hitachi Energy's core values safety and integrity, which means taking responsibility for your own actions while caring for your colleagues and the business.
• Your Background:
• BE / B.Tech in Computer Science, Data Science, or related discipline and at least 5 years of related working experience.
• 5 years of data engineering experience, with understanding lake house architecture, data integration framework, ETL/ELT pipeline, orchestration/monitoring, star schema data modeling.
• 5 years of experience with Python/PySpark and SQL(Proficient in PySpark, Python, and Spark SQL) 2-3 years of hands-on data engineering experience using Databricks as the main tool (meaning >60% of their time is using Databricks instead of just occasionally).
• 2-3 years of hands-on experience with different Databricks components (DLT, workflow, Unity catalog, SQL warehouse, CI/CD) in addition to using notebooks.
• Experience in Microsoft Power BI.
• Additional Insights:
• Basic understanding of Machine Learning algorithms.
• Enhanced Data Processing : AI/ML can automate and improve data processing tasks, making them more efficient and accurate.
• Predictive Analytics : Integrating AI/ML can help in building predictive models that can forecast trends and outcomes, providing valuable insights for decision-making.
• Personalization : AI/ML can be used to create personalized experiences for users by analyzing their behavior and preferences.
• Automation : AI/ML can automate repetitive tasks, freeing up time for more complex and creative work.
• Ability to quickly grasp concepts from a field that is not one's core competency; A fast learning generalist capable of solving problems independently and resourceful in a matrixed corporate environment.
• Proficiency in both spoken & written English language is required.
• Apply now
• Current employees apply here

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Hitachi Energy? Share your experience