The Data Engineer and Analyst is responsible for designing, coding, and implementing new business application sources to be extracted, transformed and loaded to the data lake.
The Data Engineer and Analyst must have good listening skills to be able to understand business needs and
interpret those needs into an efficient and effective application design.
Responsibilities
Consults with users, clients and other technology groups on issues and recommends advanced programming solutions.
Accountable for all phases of development process, analysis, design, construction, testing and implementation.
Provides user and operational support on applications to business users.
Solves / works complex problems / projects through in-depth evaluation of business processes, system processes and industry standards; performs root cause analyses.
Engages in post implementation analysis of business usage to ensure successful system design and functionality.
Provides sound understanding of Big Data apps development concepts and principles and knowledge of concepts and principles in other technology areas.
Provides evaluative judgment based on analysis of factual information in complicated and unique situations.
Directly impacts the business by ensuring the quality of work provided by self and others.
Impacts own team and closely related work teams. Exhibits sound and comprehensive communication and diplomacy skills to exchange complex information
Strong leadership, interpersonal, influence, negotiation, and written and verbal communication skills required.
Utilize ad-hoc techniques to perform on-the-fly analysis of data
Managerial (team/group responsibilities)
Support the division & department deliverables
Organizational (organizational responsibilities)
Work closely with Department & Section Head to set department direction and AAM according to division and corporate vision
Work closely with all levels of staff to achieve corporate goals and objectives
Skills and Experience We Are Looking For:
Writes advanced SQL including some query tuning
Experience in the identification of data quality
Experience working with Stored Procedures
Some database design experience is helpful
Implementing data Compression techniques for space saving
Experience designing and building complete ETL processes moving and transforming data for ODS, Staging, and Data Warehousing
Experience with Informatica ETL tool
Proficient in how to deploy to and debug a production Data Lake environment
Experience with real time and batch data ingestion frameworks using (Kafka/Flink/Hudi/Hive/Clickhouse)
Hlb · Pjc-pj City