Engineer III, Software

A day in the Life:

• Lead and deliver the projects within APJ Data Engineering Team
• Perform source to target data analysis and mappings.
• Collaborate with admins, developers, data engineers and analysts for successful functionality delivery.
• Perform requirement analysis and co-ordinate with project managers and development team to drive the delivery cycle.
• Collaborate with the scrum master on product backlogs and helps with sprint planning.
• Improve/Support solutions using Pyspark/EMR, SQL and databases, AWS Athena, S3, Redshift, AWS API Gateway, Lambda, Glue and other Data Engineering technologies.
• Write Complex Queries and edit them as required for implementing ETL/Data solutions.
• Handle database migration projects from Redshift to Data Lake S3, ensuring seamless transition and minimal disruption to operations.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Your main tasks will involve using Databricks, advanced SQL, and Power BI to achieve this goal.

Keys to Success:

Education

• Bachelor's degree in Computer Science with at least 6 years of Data Engineering experience using AWS services, Pyspark/EMR.
• Having Certifications like AWS Certified Data Analytics, CCA Spark and Hadoop Developer or CCP Data Engineer is recommended.

Experience

• 6+ Years of Experience in

  Databricks, Data Analytics & Business Intelligence Solutions

  and Technical Lead.
• Full life cycle project implementation experience in AWS using Databricks Pyspark/EMR, Athena, S3, Redshift, AWS API Gateway, Lambda, Glue and other handled services
• Good experience on Power BI.
• Experience with agile development methodologies by following DevOps, Data Ops and Dev Sec Ops practices.
• Experience using Jira for task prioritization and Confluence and other tools for documentation.
• Strong analytical experience with database in writing complex queries, query optimization, debugging, user defined functions, views, indexes etc.

Knowledge, Skills, Abilities

• Strong communication skills in both written and verbal forms, with a focus on effective collaboration with others, including customers, partners, and team members.
• Excellent prioritization and problem-solving skills.
• Learn and train other team members
• Aim to meet and exceed BI Operational SLAs for Service Now incidents, Major Incidents, xMatters alerts, Employee Experience Metrics and BI application /process availability metrics.