Data Engineer - Pyspark

As a Data Engineer - Pyspark at Barclays, you will play a pivotal role in advancing our infrastructure and deployment pipelines, fostering innovation and operational excellence. Your responsibility will include leveraging state-of-the-art technology to develop and manage robust, scalable, and secure infrastructure, ensuring the seamless delivery of our digital solutions. To excel in this role, you should possess expertise in PySpark and Python, along with a strong foundation in SQL. You must exhibit proficiency in writing and debugging code, coupled with being a quick learner with exceptional analytical and problem-solving skills. Effective written and verbal communication skills are essential for this position. Additional highly valued skills may encompass knowledge in AWS cloud services like S3, Glue, Athena, Lake Formation, CloudFormation, familiarity with SCM tools such as GIT, previous experience in the banking or financial services sector, and exposure to technologies like Databricks, Snowflake, Starburst, and Iceberg. Your performance may be evaluated based on critical skills crucial for success in the role, such as risk and controls management, change and transformation facilitation, business acumen, strategic thinking, and digital and technology proficiency. The position is based in Chennai. In this role, your primary objective is to construct and uphold systems for collecting, storing, processing, and analyzing data, comprising data pipelines, data warehouses, and data lakes. This is essential to ensure the accuracy, accessibility, and security of all data. Key Accountabilities: - Develop and maintain data architecture pipelines that facilitate the transfer and processing of durable, complete, and consistent data. - Design and implement data warehouses and data lakes that can manage appropriate data volumes and velocity while adhering to necessary security measures. - Create processing and analysis algorithms tailored to the complexity and volumes of the intended data. - Collaborate with data scientists to construct and deploy machine learning models. Analyst Expectations: - Execute assigned tasks promptly and with high standards, driving continuous improvement. - Demonstrate profound technical knowledge and experience in the designated expertise area. - Lead and supervise a team, providing guidance, professional support, allocating work requirements, and coordinating resources. - Take responsibility for team operations and activities, ensuring adherence to policies and procedures, risk mitigation, and regulatory compliance. - Actively engage with stakeholders, influence decision-making, and take ownership of risk management and control enhancement. - Foster cross-functional collaboration, maintain organizational awareness, and contribute to achieving organizational objectives by resolving problems and guiding team members. In alignment with the Barclays Values of Respect, Integrity, Service, Excellence, and Stewardship, you are expected to uphold a moral compass and exhibit the Barclays Mindset of Empower, Challenge, and Drive in your conduct.,