Agile-EY-GDS Consulting-AI And DATA-Azure Manager - E

Job Profile Summary:

Provides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure. Utilizes a deep dive analytical skillset on a variety of internal and external data.

Skills:

• Advance level of expertise in Glue, EMR, Redshift, Step Functions, Foundational Services like IAM, Cloud Watch, Cloud formation, Lambda, Secrets Manager, Sagemaker, Pyspark, SQL

Qualifications:

• Minimum of six years ( 6 to 9 years ) in

  data engineering , Data analytics , programming, database administration, or data management experience.
• Graduate degree ( Bachelors Degree , BCA or Masters ) or equivalent combination of training and experience.

Communication And Team Collaboration

• Strong verbal and written communication, facilitation & relationship-building
• Be highly flexible & adaptable
• Accountability & E2E ownership

• Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing.
• Troubleshoots software and processes for data consistency and integrity. Integrates large scale data from a variety of sources for business partners to generate insight and make decisions.
• Translates business specifications into design specifications and code. Responsible for writing complex programs, ad hoc queries, and reports. Ensures that all code is well structured, includes sufficient documentation, and is easy to maintain and reuse.
• Partners with internal clients to gain an enhanced understanding of business functions and informational needs. Gains expertise in tools, technologies, and applications/databases in specific business areas and company-wide systems.
• Leads all phases of solution development. Explains technical considerations at related meetings, including those with internal clients and less experienced team members.
• Experience building AWS cloud architecture and supporting services and technologies (Eg: Glue, EMR, Step Functions, Foundational Services like IAM, Cloud Watch, Cloud formation, Lambda, Secrets Manager, Sagemaker).
• Experience building Spark data processing applications (Python, Pyspark).
• Experience with Apache airflow ( + iceberg preferred ) & SQL development and Tableau Reporting ( Preferred) .
• Experience leveraging / building Splunk dashboards.
• Experience with test automation and test-driven development practices.
• Experience with CI/CD pipeline tools like Bamboo, Bitbucket, Git hub and Terraform scripts / CFS .
• Tests code thoroughly for accuracy of intended purpose, including regression testing . Reviews end product with the client to ensure adequate understanding. Provides data analysis guidance as required.
• Designs and conducts training sessions on tools and data sources used by the team and self provisioners. Provides job aids to team members and business users.