Principal Data Engineer (IC5/IC4)

About the role

The Mail Intelligence Org develops intelligent capabilities at scale to uncover interests, reveal habits, and personalize user journeys across the Yahoo ecosystem. We manage billions of messages using advanced backend systems and algorithms, including NLP, GenAI, and ML. Our infrastructure extracts information and integrates diverse data sources to amplify key insights. We seek innovative, entrepreneurial engineers who value high standards, code quality, and world-class architectural expertise. A lot about YOU

Responsibilities

• Make an impact on Y! Mail Intelligence mission: lead the design, build and evolution of data solutions and pipelines that
• are foundational to AI/ML-powered personalization and assistive intelligence,
• scale to 220+ million monthly active users,
• are robust , self-healing, and leverage AI-driven diagnostics to minimize manual recovery efforts.
• Automated Data Governance: Establish and implement automated frameworks for data quality and compliance, ensuring integrity across petabyte-scale datasets.
• Technical Leadership and Collaboration: P rimary technical liaison between Dublin and US-based cross-functional teams to align data strategies and architectural roadmaps.
• AI-Driven Efficiency: Proactively identify and automate repetitive engineering tasks by integrating AI tools into the development lifecycle, replacing manual workflows with intelligent automation.
• Advanced Performance Optimization: Lead the continuous improvement of data workflows, balancing cost, quality, and speed through automated testing and performance tuning.
• Strategic Tradeoff Management: Exercise sophisticated judgment in managing system complexity, exercising an AI-first mindset to solve large-scale data challenges.

Requirements

• AI-Forward Mindset: You view AI as an essential partner in increasing your impact, with a demonstrated commitment to experimenting with emerging tools to accelerate delivery.
• Architectural Depth: You possess a strong foundation in computer science and data engineering, with a proven track record of managing complex cloud-native architectures at scale. We are looking for good software engineers who are experts in data .
• Technical Mastery: You are highly proficient in
• one object oriented programming language (Python / Java preferred),
• SQL,
• Spark (PySpark / Java Spark preferred)
• Architecting, designing and building batch and streaming data pipelines.
• Hadoop is a big plus.
• Data Integrity Focus: You maintain uncompromising standards for data quality, built data quality frameworks, architect systems that prevent failures before they occur.
• Collaborative Architect Mindset: You excel in working across global time zones, providing thought leadership and technical guidance to multidisciplinary project teams.
• Educational Requirement : Bachelor's degree in Computer Science, Data Engineering, or a related field (must have).
• Experience : 8+ years of professional experience in data engineering or big data processing.
• Cloud Expertise : Deep proficiency with public cloud platform (bonus points for Google Cloud Platform - GCP proficiency).
• Big Data Stack : Expert-level knowledge of stream/batch processing frameworks and data orchestration (Composer/Airflow).
• AI Proficiency: Demonstrated ability to leverage AI coding assistants (e.g., GitHub Copilot, Cursor) and LLMs (e.g., Claude, Gemini) to accelerate development velocity and optimize system architecture.
• Problem Solving : Proven ability to solve complex, multifaceted problems with innovative, scalable solutions.
• Communication : Excellent verbal and written communication skills, with the ability to influence technical direction across international teams.
• Prior experience with email systems or high-volume messaging platforms.
• Advanced experience with DataProc, Dataflow, Composer and Looker Studio for intelligent data visualization.
• Master's degree in a technical field.
• Yahoo is proud to be an equal opportunity workplace. All qualified applicants will receive considerati

Benefits

Additional Information

Yahoo Mail is the ultimate consumer inbox with hundreds of millions of users. It's the best way to access your email and stay organized from a computer, phone or tablet. With its beautiful design and lightning fast speed, Yahoo Mail makes reading, organizing, and sending emails easier than ever. Location: Dublin, Ireland (Hybrid - 2 days/week on-site) Applicants must have the legal right to work in Ireland at the time of application. A little about Us As a global email provider, Yahoo Mail is the ultimate Consumer Inbox with over 220 million monthly active users. We create technology that changes the internet while handling billions of inbound connections per day to manage trillions of messages requiring petabytes of efficient storage. Our vision is to be the best consumer email platform helping users manage the business of their lives. We are currently 100% committed to a native public cloud architecture.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Yahoo? Share your experience