Data BAU Lead

Responsibilities

• We are looking for a seasoned BAU Lead - Data Engineering to lead our data
• operations and support function in a cloud-first environment. The ideal
• candidate will bring strong technical leadership, hands-on expertise in AWS
• and Snowflake, and proven experience in managing Business-As-Usual (BAU)
• data platforms, production support, and continuous improvement initiatives.
• ▪ Develop tools to improve data flows between internal/external systems
• and the data lake/warehouse.
• ▪ Work with stakeholders to understand needs for data structure, availability,
• scalability, and accessibility.
• ▪ Build robust and reproducible data ingest pipelines to collect, clean,
• harmonize, merge, and consolidate data sources.
• ▪ Understanding existing data applications and infrastructure architecture
• ▪ Build and support new data feeds for various Data Management layers and
• Data Lakes
• ▪ Evaluate business needs and requirements.
• ▪ Support migration of existing data transformation jobs in Oracle, and MS[1]SQL to Snowflake.
• ▪ Lead the migration of the existing data transformation jobs in Oracle, Hive,
• Impala etc. into Spark, Python on Glue etc.
• ▪ Able to document the processes and steps.
• ▪ Develop and maintain datasets.
• ▪ Improve data quality and efficiency.
• ▪ Lead Business requirements and deliver accordingly.
• ▪ Collaborate with Data Scientists, Architect and Team on several Data
• Analytics projects.
• ▪ Collaborate with DevOps Engineer to improve system deployment and
• monitoring process.
• ▪ Experience in critical production support and how the BAU functions is
• preferred.
• Requirements Key Skills:
• ▪ Lead a team of data engineers in managing day-to-day BAU operations,
• production support, incident management, and platform stability.
• ▪ Strong AWS knowledge in terms of designing new architecture and
• providing optimized solutions for existing ones. (S3, Glue, DMS, MWAA,
• AMS, IAM).
• ▪ In-depth knowledge with respect to Snowflake and its architecture.
• ▪ Prefer prior Experience in BAU environment.
• ▪ Good knowledge on Airflow and MWAA.
• ▪ Hands-on experience in SQL/Python/Pyspark.
• ▪ Expertise in optimizing techniques in cloud environments.
• ▪ Should have the vision on data strategy and able to deliver the same.
• ▪ Should be able to lead the design and implementation of data management
• processes, including data sourcing, integration, and transformation.
• ▪ Able to manage and lead a team of data professionals, providing guidance,
• mentoring and foster a collaborative and innovative team culture focused on
• continuous improvement.
• ▪ To evaluate and recommend data-related technologies, tools, and
• platforms.
• ▪ Collaborate with IT teams to ensure seamless integration of data solutions.
• ▪ Should have experience in Implementing and enforcing data security
• protocols and ensure compliance with relevant regulations.
• Required Qualifications:
• ▪ At least 8+ years of Data Engineer Experience
• ▪ Bachelor qualification in a computer science or STEM (science, technology,
• engineering, or mathematics) related field.
• ▪ At least 5+ years of recent hands-on professional experience (actively
• coding) working as a Lead handling support & production issue.
• ▪ 2+ experience with large scale datasets, data lake and data warehouse
• technologies such as AWS Redshift, Google BigQuery, Snowflake. Snowflake
• is highly preferred.
• ▪ At least 3+ years of experience in ETL (AWS Glue), Amazon S3, Amazon
• RDS, Amazon Kinesis, Amazon Lambda, Apache Airflows, Amazon Step
• Functions.
• ▪ Professional experience working in an agile, dynamic and customer-facing
• environment is required.
• ▪ Understanding of distributed systems and cloud technologies (AWS) is
• highly preferred.
• ▪ Understanding of data streaming and scalable data processing is preferred
• to have.
• ▪ Strong knowledge in scripting languages like SQL ,Python, UNIX shell and
• Spark is required.
• ▪ Understanding of RDBMS, Data ingestions, Data flows, Data Integrations
• etc.
• ▪ Technical expertise with data models, data mining and segmentation
• techniques.
• ▪ Experience with full SDLC lifecycle and Lean or Agile development
• methodologies.
• ▪ Knowledge of CI/CD and GIT Deployments.
• ▪ Ability to work in team in diverse/ multiple stakeholder environment.
• ▪ Ability to communicate complex technology solutions to diverse teams
• namely, technical, business and management teams
• Soft Skills
• ▪ Ability to work in a collaborative environment and coach other team
• members on coding practices, design principles, and implementation
• patterns that lead to high quality maintainable solutions.
• ▪ Excellent communications and stake-holder management are required.
• ▪ Ability to handle senior leadership and report to them if required.
• ▪ Ability to work in a dynamic, agile environment within a geographically
• distributed team.
• ▪ Ability to focus on promptly addressing customer needs.
• ▪ Ability to work within a diverse and inclusive team.
• ▪ Technically curious, self-motivated, versatile and

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at TECHNOPALS PTE. LTD.? Share your experience