Sr. Technical Expert

Job Role AWS Data Engineering

Roles and Responsibility -

• Platform Monitoring: Monitor the query/jobs queue, ensuring no queries exceed a waiting time of 1 second.
• Proactively perform necessary adjustments to maintain optimal performance.
• Incident Management: Monitor and respond promptly to related incidents logged in ServiceNow, ensuring resolution within agreed SLAs and provide root cause analysis.
• AWS DevOps and IaC: Build and support pipelines.
• Manage infrastructure as code through Cloud formation/Terraform/any other IaC, covering S3, Glue, Lake formation, Lambda.
• Community Engagement: Actively monitor and engage with the Teams community channel, providing timely responses and following up queries to their resolution.
• AWS Infrastructure Support: Support and maintain AWS infrastructure including CloudFormation/Terraform scripts, AWS Code Deploy, Lambda functions, event triggers, and Kubernetes/EKS environments.
• Data Engineering Support: Assist development teams with data ingestion, transformation and job scheduling (Glue, EMR, PySpark, SQL).
• Reporting: Generate monthly reports summarizing issues, incidents, and response patterns, highlighting opportunities for process improvement or automation.

Experience - 8 to 12 yrs

Top Skills

• Essential Technical Skills: Mandatory: Advanced SQL performance tuning and optimisation for large?scale analytics workloads
• Expert-level PySpark development and troubleshooting on AWS services such as EMR and AWS Glue
• Proficient Python scripting for automation, data pipelines, and tooling (boto3, PySpark helpers)
• Infrastructure as Code on AWS using Terraform (authoring, reviewing, and deploying stacks) or any other IaC
• Operational experience with core AWS data services: S3, Glue, Athena, Airflow, EMR, Lambda, IAM
• Familiarity with monitoring and alerting via Amazon CloudWatch, Prometheus, or Grafana
• CI/CD pipelines Gitlab,any other rep[o management tools
• Preferred and Desirable Exposure to cost?optimization and security best practices (VPC endpoints, encryption, tagging, lifecycle policies)
• Experience implementing data?governance controls and role?based access in AWS Lake Formation
• Desired Traits Strong sense of ownership and proactive approach Excellent analytical and troubleshooting skills Effective communicator and collaborative team player Eagerness to learn and provide knowledge transfer on AWS Kubernetes services as needed.