Data Engineer

About the role

The Data Engineering team is seeking a Data Engineer. The team's mission is to create and publish high quality public datasets, build transparent and automated data pipelines using open-source technologies, develop a data ecosystem by offering comprehensive documentation and analytics resources, and bring people together across teams and agencies to share data and learn from each other. The team is continually evolving, and this new hire will have the ability and support to improve existing infrastructure and processes and drive forward new projects and initiatives. The ideal candidate will be excited about advancing the adoption of new technologies in DCP and City government and passionate about data production, quality, and accessibility. Under limited supervision, the Data Engineer's primary areas of responsibility include, but are not limited to: o Designing, deploying, and scaling of current infrastructure to ensure that Data Engineering maintains and improves existing processes, and expands its abilities and offerings. o Understanding the technologies used by the team and recommending new technologies. Ideating, designing, and implementing improvements to products and processes to ensure that the team is using the best tools for the job. o Building Data Engineering data products, which involves: o operating data ingestion pipelines and managing data storage, o running scripts that transform input data by standardizing, geocoding, merging, aggregating, and performing basic spatial manipulations, and o reviewing outputs by writing custom scripts and/or using established QAQC tools to identify errors, inconsistencies, and edge cases in datasets. o Designing new data products from concept to completion, working on specifications with product owners, creating technical designs with team members, helping to plan sprints, contributing to documentation, and delivering a finished product. o Collaborating with other data engineers in sprint planning, design, code review, and pair programming. o Learning and wrangling complex legacy data systems and bringing the