Senior Associate - Data Engineering

Key Responsibilities

• Interpreting and analyzing business requirements and converting them into high and low level designs.
• Designing, developing, configuring, testing and deploying cloud automation for Finance business unit using tools such as CloudFormation, Terraform, Ansible etc. while following the capability domains Engineering standards in an Agile environment
• End-to-end ownership of developing, configuring, unit testing and deploying developed code with quality and minimal supervision.
• Work closely with customers, business analysts and technology & project team to understand business requirements, drive the analysis and design of quality technical solutions that are aligned with business and technology strategies and comply with the organization's architectural standards.
• Understand and follow-up through change management procedures to implement project deliverables.
• Coordinate with support groups such as Enterprise Cloud Engineering teams, DevSecOps, Monitoring to get issues resolved with a quick turnaround time.
• Work with data science user community to address an issue in ML(machine learning) development life cycle.

Required Qualifications

• Bachelors or Masters degree in Computer Science or similar field
• 4 to 7 years of experience in automation on a major cloud (AWS, Azure or GCP)
• Experience in infrastructure provisioning using Ansible, AWS Cloud formation or Terraform, Python or PowerShell
• Working knowledge of AWS Services such as EC2, Cloud Formation, IAM, S3, EMR, ECS/EKS etc.
• Advanced proficiency in PySpark for distributed data processing and transformation.
• Working knowledge of CI/CD tools and containers.
• Experience in hadoop administration in resolving hive/spark related issues.
• Proven understanding of common development tools, patterns and practices for the cloud.
• Experience writing automated unit tests in a major programming language
• Proven ability to write quality code by following best practices and guidelines.
• Strong problem-solving, multi-tasking and organizational skills.
• Good written and verbal communication skills.
• Demonstrable experience of working on a team that is geographically dispersed.
• Experience with SQL and query optimization across large datasets.

Preferred Qualifications

• Experience with managing Hadoop platform and good in debugging hive/spark related issues.
• Cloud certification (AWS, Azure or GCP)
• Knowledge of UNIX/LINUX shell scripting