Design, develop, and implement robust PySpark‑based ETL frameworks with error handling and fault tolerance in distributed cluster environments.
Perform data enquiry, data analysis and data sourcing in Relational and Big Data database environments.
Develop a thorough understanding of technology systems, data stores, data pipelines, job streams, and downstream data consumers.
Translate business requirements into data models and program design specifications
Modify ETL and related programs to implement changes driven by evolving business requirements
Respond to queries from business analysts and developers regarding data access, query design, and performance optimization.
Troubleshoot data issues and collaborate with data providers and upstream teams to identify root causes and drive resolution. Provide Level‑3 (L3) production support as needed.
Use AI‑enabled tools to accelerate data mapping, data lineage creation, and data profiling, improving turnaround time and overall data quality.
Requirements
Experience with coding in all: Pyspark, Python, SQL
Strong knowledge of relational databases and data modeling
Strong SQL query-writing skills
Strong analytical and troubleshooting skills.
At least 5 years financial industry technology experience and knowledge of financial services data
Good verbal and written communication skills
Hands-on Experience with Graph Database/Stardog
Experience with coding Autosys job definitions
Experience with Data Modeling tools (e.g., Power Designer)
Experience in Linux Shell
Experience in Java Development
Experience and Understanding of Data Quality Rules and Implementation
WHAT YOU CAN EXPECT FROM MORGAN STANLEY:
To learn more about our offices across the globe, please copy and paste https://www.morganstanley.com/about-us/global-offices into your browser.
Expected base pay rates for the role will be between 90,000 and 150,000 per year at the commencement of employment. However, base pay if hired will be determined on an individualized basis and i
Benefits
Vision insurance
Additional Information
In the Technology division, we leverage innovation to build the connections and capabilities that power our Firm, enabling our clients and colleagues to redefine markets and shape the future of our communities.
This is a Principal Software Engineering position at Associate level which is part of the job family responsible for developing and maintaining software solutions that support business needs.
Since 1935, Morgan Stanley is known as a global leader in financial services, continuously evolving and innovating to better serve our clients and our communities in more than 40 countries around the world
Morgan Stanley · New York, NY