Senior Data Engineer

Responsibilities

• Own end-to-end data flows from requirements and architecture through implementation and production operations, including Data acquisition, Data set acceptance criteria, and Data Science integration.
• Design and build scalable batch and real-time data pipelines and lakehouse solutions with a focus on large-scale data processing.
• Take responsibility to explore technologies to scale up the Data ecosystem to handle rapid Big Data growth.
• Partner with Data Science to productionize ML/AI workloads and ensure smooth integration into products.
• Collaborate with cloud, DevOps, application, and client teams to deliver robust, secure, and scalable solutions that solve meaningful business problems.
• Evaluate and adopt new technologies and patterns to evolve the data ecosystem as scale and complexity grow.
• 3+ years of hands-on Data Engineering building and operating production-grade Data Systems and Pipelines (Data-Intensive, Distributed Processing, Databases).
• B.Sc. / M.Sc. in Computer Science, Computer Engineering, or equivalent.
• Proficiency in Python; working proficiency in Scala.
• Strong expertise with at least one major cloud provider (AWS, Azure, or GCP).
• Strong experience with Big Data processing (Spark, DataBricks) and event streaming (Kafka).
• Experience with orchestration and platform tooling such as Airflow; ability to build maintainable DAGs and operationalize workflows.
• Strong SQL skills and experience with data storage systems plus at least one of: Data Lake/Lakehouse, columnar DB, or NoSQL systems.
• Hands-on experience with containers and Kubernetes (Helm is a plus) and modern CI/CD practices.
• Familiarity with LLM workflow frameworks (LangChain/LangGraph).
• Proven experience designing, building, and owning production-grade data pipelines (batch and/or streaming), including reliability, backfills, and SLA-driven delivery.
• Ability to learn new technologies and work in a dynamic fast-paced environment.
• Result-driven, pragmatic, and innovative.
• Strong analytical skills, with an open and proactive mindset to investigate, learn, and propose solutions; highly self-driven and self-taught.
• Strong English communication skills, both written and verbal.

Requirements

• Experience with Delta Lake and/or Apache Iceberg; ML lifecycle tools such as MLflow.
• Experience with Pandas/Polars and building data services/APIs (e.g., FastAPI).
• Experience building LLM-powered agents / chat assistants (RAG, tool/function calling, workflow or multi-agent orchestration), using modern frameworks and platforms.
• Infrastructure as Code (Terraform/Pulumi/CloudFormation) and cloud security fundamentals (IAM, secrets, encryption).
• Experience with observability tooling (metrics/logging/tracing) and cost/performance optimization for distributed workloads.
• Experience building applications with React and Node.js.

Benefits

Additional Information

As a Data Software Engineer, you will be responsible for building new data solutions for our rapidly expanding customer base and working with the top data ingestion technologies, working with a team of amazing, diverse-minded, and bright people who make an impact, generate creative & innovative ideas, and take on new perspectives.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Nielseniq? Share your experience