Data Engineer (Hadoop, Scala and Databricks) - Assistant Manager

As a Data Engineer at our company, you will collaborate with various teams to meet our Data Platform needs. Your responsibilities will include designing, implementing, and maintaining a secure and scalable infrastructure platform across AWS/Azure and our Data Center. You will also be responsible for ensuring that internal and external SLAs are met, monitoring system-centric KPIs, and working on data lake, data warehouse, and stream processing systems. Your role will involve creating a unified query engine, multi-model databases, analytics extracts, reports, dashboards, and visualizations. Additionally, you will design systems for high availability, high throughput, data consistency, security, and end user privacy. Mentoring other engineers, promoting software engineering best practices, and coming up with solutions for scaling data systems will be part of your responsibilities. Key Responsibilities: - Collaborate with teams to meet Data Platform needs - Design, implement, and maintain a secure and scalable infrastructure platform - Ensure internal and external SLAs are met and system-centric KPIs are monitored - Work on data lake, data warehouse, and stream processing systems - Create a unified query engine, multi-model databases, analytics extracts, reports, dashboards, and visualizations - Design systems for high availability, high throughput, data consistency, security, and end user privacy - Mentor other engineers and promote software engineering best practices - Come up with solutions for scaling data systems Qualifications Required: - Knowledge of at least one public cloud platform: AWS, Azure, or GCP - Proficiency in a programming language such as Python, Scala, or SQL - Understanding of end-to-end data analytics workflow - Hands-on experience in Databricks The company's technology function, Global Technology Services (GTS), plays a vital role in driving the company's digital transformation and expanding business capabilities. As a Data Engineer, your role is important in delivering data and insights to clients using advanced technologies such as cloud, artificial intelligence, and robotics process automation. Joining the team will provide you with opportunities to grow your technical skills, solve real problems, and contribute to the industry. State Street is committed to creating a great work environment where innovation and technology skills are valued. We offer competitive benefits packages, flexible work programs, and development opportunities to help employees reach their full potential. (State Street is an equal opportunity and affirmative action employer. Visit StateStreet.com/careers for more information.),