AWS Pyspark Developer

AWS PySpark Developer

• Design, build, and optimize robust data pipelines and data architecture on the AWS cloud platform.
• Wrangle, explore, and analyze large datasets to identify trends, answer business questions, and pinpoint areas for improvement.
• Develop and maintain a next-generation analytics environment, providing a self-service, centralized platform for all data-centric activities.
• Formulate and implement distributed algorithms for effective data processing and trend identification.
• Configure and manage Identity and Access Management (

  IAM

  ) on the AWS platform.
• Collaborate with stakeholders to understand data requirements and deliver effective solutions.

Required Skills & Experience:

• 2-8 years of experience as a Data Engineer or Developer.
• Proven experience building and optimizing data pipelines on AWS.
• Proficiency in scripting with

  Python

  .
• Strong working knowledge of:

• Big Data Tools:

  AWS Athena.

• Relational & NoSQL Databases:

  AWS Redshift and PostgreSQL.

• Data Pipeline Tools:

  AWS Glue, AWS Data Pipeline, or AWS Lake Formation.

• Container Orchestration:

  Kubernetes, Docker, Amazon ECR/ECS/EKS.
• Experience with wrangling, exploring, and analyzing data.
• Strong organizational and problem-solving skills.

Preferred Skills:

• Experience with machine learning tools (

  SageMaker, TensorFlow

  ).
• Working knowledge of stream processing (

  Kinesis, Spark-Streaming

  ).
• Experience with analytics and visualization tools (

  Tableau, Power BI

  ).
• Knowledge of optimizing

  AWS Redshift

  performance.
• Familiarity with

  SAP Business Objects (BO)

  .