Design, develop, and maintain robust data pipelines to ingest, transform, and load data into multiple data stores
Write efficient, scalable, and maintainable code, primarily using Python and PySpark
Collaborate closely with internal stakeholders to understand data requirements and deliver effective solutions
Resolve day-to-day technical challenges independently, without the need for constant guidance
Provide proactive, professional, and reliable technical support when required
Produce high-quality technical documentation to support delivered solutions
Actively participate in Agile planning, design, and refinement sessions
Contribute to continuous improvement of Agile processes to enhance team productivity and effectiveness
Participate in code peer reviews to improve code quality and ensure adherence to development standards
Engage in regular team meetings to align on priorities, highlight risks, share decisions, and resolve issues
Support, enhance, and maintain CI/CD pipelines for data solutions
Stay current with emerging technologies and maintain strong technical awareness within the data engineering domain
Team Working
Be an active member of the VDX Data team, contribute to team dynamics, ways of working and assisting with improvement opportunities
Provide regular and accurate reports of progress to Technical leads
Build strong relationships with stakeholders with a view to providing high-value solutions within the business whilst keeping communication channels open at all times
Maintain strong technical awareness and familiarity with new and upcoming technologies around Data Integration and Business Intelligence Analysis.
Stay abreast of the industry and participate in external communities in order to keep up to date and offer the most informed position when defining or consulting on solution design
Availability and flexibility of working hours, as standard 11:00 - 7:30 pm would be expected.
Knowledge, skills and experience required
Core Skills
Bachelor's degree in Engineering, or equivalent practical experience
2-4 years of experience in a Data Engineering role or working with comparable data systems and technologies
Hands-on experience with Agile development methodologies such as Scrum or Kanban
Experience working within distributed, multi-team delivery environments
Technical Skills
Strong understanding of modern data architectures and data modelling principles
Advanced programming expertise in Python and PySpark
Extensive hands-on experience with AWS services, including Glue, Lambda, S3, SES, SNS, Athena, and EMR
Solid working knowledge of Databricks applications
Strong proficiency in SQL for data querying and transformation
Familiarity with CI/CD practices for building and deploying data pipelines
Experience using source control systems such as GitHub Enterprise
Personal Skills
Proven ability to build and manage effective stakeholder relationships
Excellent written and verbal communication skills in English
Enthusiastic, approachable, and confident communicator, capable of presenting and guiding others
Ability to bridge communication gaps between technical teams and business stakeholders
Strong passion for technology with a willingness to learn and adopt new tools, frameworks, and methodologies
Ability to thrive in a fast-paced and evolving environment
Self-motivated with the ability to work independently and take initiative
Flexible and willing to take on a variety of roles and responsibilities
About us: Our story
The group provides unrivalled insight and expertise around affluent consumers and frequent travellers,
Benefits
Vision insuranceFlexible schedule
Additional Information
Recruiter for this role:
Britney Dias
Purpose of the job
The Data Science and Engineering Team works within the broader Collinson Central Data Team but is directly related to our Valuedynamx division, the division is in charge of managing, operating and servicing loyalty ecommerce programs. This involves working across multiple datasets (of varying scale and complexity), multiple teams and businesses to build appropriately styled, data-driven solutions to a commercial-product level.
This role focuses on collecting, parsing, managing and analysing large sets of data to turn information into insights using multiple platforms. The Data Engineer will decide on the needed infrastructure and software design needs of the department and provide recommendations for building solutions. The Data Engineer should be able to develop prototypes and proof of concepts for the selected solutions.
Collinsongroup · Mumbai, India