3 Emr Serverless Jobs

As a Principal Data Engineer at FTSE Russell, you will play a crucial role in leading the development of foundational components for a lakehouse architecture on AWS. Your primary responsibility will involve driving the migration of existing data processing workflows to the new lakehouse solution. Collaboration across the Data Engineering organization is essential as you design and implement scalable data infrastructure and processes using cutting-edge technologies such as Python, PySpark, EMR Serverless, Iceberg, Glue, and Glue Data Catalog. Success in this role hinges on your deep technical expertise, exceptional problem-solving skills, and your ability to lead and mentor within an agile team. You will lead complex projects autonomously, setting a high standard for technical contributions while fostering an inclusive and open culture within development teams. Your strategic guidance on best practices in design, development, and implementation will ensure that solutions meet business requirements and technical standards. In addition to project leadership and culture building, you will be responsible for data development and tool advancement. This includes writing high-quality, efficient code, developing necessary tools and applications, and leading the development of innovative tools and frameworks to enhance data engineering capabilities. Your role will also involve solution decomposition and design leadership, working closely with architects, Product Owners, and Dev team members to decompose solutions into Epics while establishing and enforcing best practices for coding standards, design patterns, and system architecture. Stakeholder relationship building and communication are crucial aspects of this role, as you build and maintain strong relationships with internal and external stakeholders, serving as an internal subject matter expert in software development. To excel as a Principal Data Engineer, you should possess a Bachelor's degree in computer science, Software Engineering, or a related field. A master's degree or relevant certifications such as AWS Certified Solutions Architect or Certified Data Analytics would be advantageous. Proficiency in advanced programming, system architecture, and solution design, along with key skills in software development practices, Python, Spark, automation, CI/CD pipelines, cross-functional collaboration, technical leadership, and AWS Cloud Services are essential for success in this role. At FTSE Russell, we champion a culture committed to continuous learning, mentoring, and career growth opportunities while fostering a culture of inclusion for all employees. Join us in driving financial stability, empowering economies, and creating sustainable growth as part of our dynamic and diverse team. Your individuality, ideas, and commitment to sustainability will be valued as we work together to make a positive impact globally.,

You will be responsible for leading the development of foundational components for a lakehouse architecture on AWS and overseeing the migration of existing data processing workflows to the new lakehouse solution. Your role will involve collaborating with the Data Engineering team to design and implement scalable data infrastructure and processes using technologies such as Python, PySpark, EMR Serverless, Iceberg, Glue, and Glue Data Catalog. The primary objective of this position is to ensure a successful migration and establish robust data quality governance across the new platform to enable reliable and efficient data processing. To excel in this position, you will need to demonstrate deep technical expertise, exceptional problem-solving skills, and the ability to lead and mentor within an agile team. Your key responsibilities will include leading complex projects independently, fostering an inclusive and open culture within development teams, and setting high standards for technical contributions. You will provide strategic guidance on best practices in design, development, and implementation to ensure that solutions meet business requirements and technical standards. Additionally, you will be involved in writing high-quality, efficient code, developing tools and applications to address complex business needs, and leading the development of innovative tools and frameworks to enhance data engineering capabilities. Collaboration with architects, Product Owners, and Dev team members will be essential to decompose solutions into Epics, lead the design and planning of these components, and drive the migration of existing data processing workflows to the Lakehouse architecture leveraging Iceberg capabilities. You will establish and enforce best practices for coding standards, design patterns, and system architecture, utilizing existing design patterns to develop reliable solutions while also recognizing when to adapt or avoid patterns to prevent anti-patterns. In terms of qualifications and experience, a Bachelor's degree in computer science, Software Engineering, or a related field is essential. A master's degree or relevant certifications such as AWS Certified Solutions Architect or Certified Data Analytics is advantageous. Proficiency in advanced programming, system architecture, and solution design is required. You should possess key skills in advanced software development practices, Python and Spark expertise, automation and CI/CD pipelines, cross-functional collaboration and communication, technical leadership and mentorship, domain expertise in AWS Cloud Services, and quality assurance and continuous improvement practices. Your role would involve working within a culture committed to continuous learning, mentoring, career growth opportunities, and inclusion for all employees. You will be part of a collaborative and creative environment where new ideas are encouraged, and sustainability is a key focus. The values of Integrity, Partnership, Excellence, and Change underpin the organization's purpose of driving financial stability, empowering economies, and enabling sustainable growth. LSEG offers various benefits, including healthcare, retirement planning, paid volunteering days, and wellbeing initiatives.,

Company Overview: ? FTSE Russell, part of the London Stock Exchange Group, is an essential index partner for a changing world, providing category-defining indices across asset classes and investment objectives to create new possibilities for the global investment community. FTSE Russells expertise and products are used extensively by institutional and retail investors globally. Job Summary: ?In this key leadership role, you will lead the development of foundational components for a lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new lakehouse solution. You will work across the Data Engineering organisation to design and implement scalable data infrastructure and processes using technologies such as Python, PySpark, EMR Serverless, Iceberg, Glue and Glue Data Catalog. The main goal of this position is to ensure successful migration and establish robust data quality governance across the new platform, enabling reliable and efficient data processing. Success in this role requires deep technical expertise, exceptional problem-solving skills, and the ability to lead and mentor within an agile team. Key Accountabilities: Project Leadership and Culture Building: Leads complex projects autonomously, fostering an inclusive and open culture within development teams. Sets a high standard for technical contributions while promoting a collaborative environment that encourages knowledge sharing and innovation. Technical Expertise and Advisory: Demonstrates profound technical knowledge of AWS data services and engineering practices. Provides strategic guidance on best practices in design, development, and implementation, ensuring solutions meet business requirements and technical standards. Data Development and Tool Advancement: Writes high-quality, efficient code and develops necessary tools and applications to address complex business needs. Leads the development of innovative tools and frameworks to enhance data engineering capabilities. Solution Decomposition and Design Leadership: Collaborates closely with architects, Product Owners, and Dev team members to decompose solutions into Epics, leading the design and planning of these components. Drive the migration of existing data processing workflows to the Lakehouse architecture, leveraging Iceberg capabilities. Establish and enforce best practices for coding standards, design patterns, and system architecture. Utilizes existing design patterns to develop reliable solutions, while also recognizing when to adapt or avoid patterns to prevent anti-patterns. Stakeholder Relationship Building and Communication: Builds and maintains strong relationships with internal and external stakeholders, collaborating across teams. Serves as an internal subject matter expert in software development, advising stakeholders on technical issues and best practices. Applies deep technical expertise to assess complex challenges and propose strategic solutions. Lead technical discussions, mentor team members, and foster a culture of continuous learning and innovation. Qualifications and Experience: Bachelors degree in computer science, Software Engineering, or a related field is essential. A masters degree or relevant certifications (e.g., AWS Certified Solutions Architect, Certified Data Analytics) is advantageous. Advanced Programming Proficiency:Deep technical knowledge of data engineering solutions and practices. Implementation of data pipelines using tools like AWS Glue, AWS Lambda, and AWS Step Functions. Proficient in Python and familiar with a variety of development technologies. This knowledge enables the Principal Data Engineer to adapt solutions to project-specific needs, apply best practices, and identify when patterns are appropriate or should be avoided. System Architecture and Solution Design: Extensive experience in software architecture and solution design, including microservices, distributed systems, and cloud-native architectures. Capable of breaking down large-scale projects into manageable components, ensuring scalability, security, and alignment with strategic objectives. Key Skills: Advanced Software Development Practices: Demonstrates mastery of best practices in software development, including knowledge of object-oriented programming, functional programming, and design patterns. Skilled at implementing complex coding structures and promoting efficient, maintainable code across projects. Advanced expertise in Python and Spark: Specialized expertise in Python and Spark, with a deep focus on ETL data processing and data engineering practices. Ability to provide technical direction, set high standards for code quality and optimize performance in data-intensive environments. Knowledge of additional programming languages and development tools provides flexibility and adaptability across varied data engineering projects. Automation and CI/CD Pipelines: Skilled in leveraging automation tools and Continuous Integration/Continuous Deployment (CI/CD) pipelines to streamline development, testing, and deployment. Experienced in setting up and optimising CI/CD processes to ensure rapid, high-quality releases and minimise manual intervention. Cross-Functional Collaboration and Communication: Exceptional communicator who can translate complex technical concepts for diverse stakeholders, including engineers, product managers, and senior executives. Skilled in building alignment and driving consensus, ensuring that technical decisions support broader business goals. Technical Leadership and Mentorship: Provides thought leadership within the engineering team, setting high standards for quality, efficiency, and collaboration. Experienced in mentoring engineers, guiding them in advanced coding practices, architecture, and strategic problem-solving to enhance team capabilities. Domain Expertise in AWS Cloud Services: Solid understanding of AWS services and cloud solutions, particularly as they pertain to data engineering practices. Familiar with AWS solutions including IAM, Step Functions, Glue, Lambda, RDS (e.g., DynamoDB, Aurora Postgres), SQS, API Gateway, Athena. Quality Assurance and Continuous Improvement: Proficient in quality assurance practices, including code reviews, automated testing, and best practices for data validation. Committed to continuous improvement, implementing methods that enhance data reliability, performance, and user satisfaction. Bonus Skills: Financial Services expertise preferred, working with Equity and Fixed Income asset classes and a working knowledge of Indices. Experienced in implementing and optimizing CI/CD pipelines. Skilled at setting up processes that enable rapid, reliable releases, minimizing manual effort and supporting agile development cycles. Other: LSEG champions a culture committed to the growth of individuals through continuous learning, mentoring and career growth opportunities. LSEG champions a culture of inclusion for all employees that respects their individual strengths, views, and experiences. We believe that our differences enable us to be a better team one that makes better decisions, drives innovation, and delivers better business results. Diversity is a core value at LSEG. We are passionate about building and sustaining an inclusive and equitable working and learning environment for all. We believe every member on our team enriches our diversity by exposing us to a broad range of ways to understand and engage with the world, identify challenges, and to discover, craft and deliver solutions. LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth. Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership , Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions. Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce. You will be part of a collaborative and creative culture where we encourage new ideas and are committed to sustainability across our global business. You will experience the critical role we have in helping to re-engineer the financial ecosystem to support and drive sustainable economic growth. Together, we are aiming to achieve this growth by accelerating the just transition to net zero, enabling growth of the green economy and creating inclusive economic opportunity. LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives. We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyones race, religion, colour, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs. Please take a moment to read this privacy notice carefully, as it describes what personal information London Stock Exchange Group (LSEG) (we) may hold about you, what its used for, and how its obtained, your rights and how to contact us as a data subject. If you are submitting as a Recruitment Agency Partner, it is essential and your responsibility to ensure that candidates applying to LSEG are aware of this privacy notice. Show more Show less