Senior Data Engineer (Databricks)

About the role

We produce globally recognized brands and we grow the best business leaders in the industry. With a portfolio of trusted brands as diverse as ours, it is paramount our leaders can lead with courage the vast array of brands, categories and functions. We serve consumers around the world with one of the strongest portfolios of trusted, q

Responsibilities

• Pipeline Engineering
• Design, build, and continuously improve pipelines from SAP to downstream
• Develop and continuously improve on Kafka / Confluent topics, connectors, schemas (Avro / Schema Registry), and stream-processing jobs.
• Build automation for deployment, validation, refresh, and reconciliation.
• Tune capture and integrate jobs for throughput, lag, and resource efficiency.
• Platform & DevOps
• Deploy and manage Kafka and HVR components on AKS / Kubernetes and Azure infrastructure.
• Maintain CI/CD pipelines (GitHub Actions / Azure DevOps) and IaC (Terraform / Bicep).
• Enforce security, RBAC, secrets management (Key Vault), and network segmentation.
• Collaboration & Governance
• Partner with SAP Basis, Network, Security
• Author runbooks, architecture diagrams, and knowledge-base articles.
• Contribute to data-quality and contract-testing frameworks. Experience
• 5+ years in data engineering or platform engineering, with 3+ years in streaming / CDC pipelines preferred
• Hands-on production experience with CDC tool preferred (ex. HVR, Debezium)
• Hands-on production experience with Apache Kafka or Confluent Platform (topics, partitions, consumer groups, Schema Registry).
• Job Qualifications

Requirements

• 5+ years in data engineering or platform engineering, with 3+ years in streaming / CDC pipelines preferred
• Hands-on production experience with CDC tool preferred (ex. HVR, Debezium)
• Hands-on production experience with Apache Kafka or Confluent Platform (topics, partitions, consumer groups, Schema Registry).
• Technical Skills
• Strong SQL (HANA, Spark SQL, or T-SQL) and understanding of SAP table structures (ACDOCA, BSEG, MARA, etc.).
• Proficiency in Python for automation, REST clients, and data tooling.
• Linux administration and shell scripting (Bash / PowerShell).
• Kubernetes / AKS workload deployment and troubleshooting.
• Cloud experience: Azure preferred (Storage, Key Vault, Monitor, Event Hubs); AWS/GCP equivalent acceptable.
• Observability stack: Grafana, Prometheus, log aggregation (ELK / Splunk / Log Analytics).
• Git and CI/CD pipelines (GitHub Actions, Azure DevOps, or Jenkins).
• Concepts
• Log-based vs trigger-based CDC; exactly-once vs at-least-once semantics.
• Schema evolution, backward/forward compatibility, contract testing.
• Event-driven and lakehouse (medallion) architecture patterns.
• Soft Skills
• Strong written communication and documentation discipline.
• Comfort with on-call and production-incident ownership.
• Cross-functional collaboration in a regulated enterprise environment.
• Preferred Qualifications (Nice-to-Have)
• Direct experience with Fivetran HVR6 (REST API, Hub/Agent, channel design).
• SAP S/4HANA / ECC functional knowledge (O2C, P2P, R2R) or basic ABAP / CDS view literacy.
• Databricks / Delta Lake / Unity Catalog development experience.
• Kafka Streams, ksqlDB, or Apache Flink.
• Terraform / Bicep infrastructure-as-code.
• Snowflake or other cloud data warehouse experience.
• Data quality frameworks (Great Expectations or custom DQ engines).
• Scala / Java for stream-processing internals.
• Prior experience operating 24/7 mission-critical replication for finance or supply-chain systems.

Benefits

Additional Information

Job Location WARSAW PLANT & GO Job Description We are seeking a Senior Data Engineer (Databricks) to design, operate, and continuously improve our SAP data replication platform. The platform streams change data from SAP S/4HANA / ECC sources into downstream like Databricks. This is used across analytics, finance, supply chain, and operations. You will own end-to-end pipeline engineering roadmap.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Procter & Gamble? Share your experience