Sr Staff - Data Platform Engineer

About the role

As a Sr. Staff Data Platform Engineer , you will be the primary architect and visionary for the core data infrastructure that powers the entire enterprise. This is a "platform-as-a-product" role; your mission is to build the internal foundation-creating the high-performance engines, abstraction layers, and self-service frameworks that enable hundreds of other engineers to move faster. You will operate at the intersection of Systems Programming and Data Engineering , solving "hard-tech" problems like automated schema evolution, multi-engine compute optimization, and global data discovery. As a top-level technical individual contributor, you will bridge the gap between long-term business strategy and deep-kernel technical execution, ensuring WEX's data platform remains a competitive advantage. Is this role for you? YES, if: You are a Backend/Software Engineer who loves solving "Big Data" problems, building APIs for data discovery, and optimizing distributed systems. NO, if: Your primary expertise is writing SQL queries, building Tableau dashboards, or managing ETL workflows without deep experience in Java or Python system architecture.

Responsibilities

• Architectural Sovereignty: Define the 3-5 year technical roadmap for the Data Lakehouse. You aren't just using tools; you are deciding how storage, compute, and metadata layers (e.g., Apache Iceberg, Snowflake Horizon or Databricks Unity Catalog ) interact at an elemental level.
• Platform-as-a-Product: Build internal SDKs, CLI tools, and automated orchestration frameworks. Your goal is to abstract away cloud complexity via Control Planes and Custom Operators , allowing Data Engineers to focus on business logic rather than infrastructure boilerplate.
• Internal R&D: Prototype and benchmark emerging technologies (e.g., specialized Spark extensions ) to keep the platform at the bleeding edge of performance and cost-efficiency.
• Global Governance & Security: Architect "compliance-by-design" systems. Automate data lineage, PII masking, and fine-grained access control across petabyte-scale environments without sacrificing developer velocity.
• AI Governance Lake: Facilitate real-time data ingestion by implementing streaming support for Open Telemetry data from AI Agents into the Data Lake, drive the development of advanced AI evaluation metrics and reporting.
• Engineering Excellence & Influence: Set the gold standard for code quality and system design across the company. You will lead Cross-Functional Architecture Reviews and serve as the final escalation point for the most complex system outages or performance bottlenecks.
• Organizational Mentorship: Beyond individual mentoring, you will foster an "Engineering Community," influencing the hiring bar and professional development paths for the entire data engineering organization.
• Qualifications & Experience
• Experience: 15+ years in software engineering and distributed systems, with at least 4 years in a principal or staff-level capacity leading platform-scale initiatives.
• Core Technical Competencies (Software Engineering Focus):
• Strong Software Foundations: Strong fundamentals on software engineering, system architecture, and scalable production applications (Algorithms, Data Structures, and System Design).
• Experience in the Java/J2EE ecosystem (Spring Boot, Microservices) and python. We are looking for a developer who writes clean, testable, and high-performance code, not just scripts.
• Data as a Product: Experience building the platforms / framework engines and APIs that power data movement, rather than just building the ETL/ELT pipelines themselves.
• Data Lakehouse Mastery:
• Deep internal knowledge of Apache Iceberg, Hudi, or Delta Lake (metadata management, manifest files, and compaction strategies).
• Experience contributing to or deeply customizing open-source data projects (e.g., Spark, dbt ).
• Extensive experience designing and building high-throughput, fault-tolerant data pipelines and orchestration frameworks that ensure robust data transformations and high data quality.
• Cloud & Infrastructure:
• Extensive experience with cloud architecture and services, including AWS (S3, EMR, Kubernetes, Lambda) and Azure .
• Deep understanding of CI/CD automation, modern development tools, Git Actions, Terraform and frameworks .
• AI-Driven Development & Productivity:
• AI Native Development: Experience leveraging AI Code Gen platforms into software development lifecycle (SDLC) to automate code generation, reviews, generate unit tests, and perform root-cause analysis of system failures.
• LLM-Ops for Platform: Ability to architect the infrastructure required to support AI Agent development by enabling vector database integration.
• Leadership & Vision:
• Proven track record of "leading by influence" -driving adoption of new technologies across multiple autonomous teams.
• Ability to communicate complex architectural trade-offs (e.g., "Latency vs. Consistency" or "Build vs. Buy") to C-suite executives and junior engineers

Benefits

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at WEX Inc? Share your experience