Data Engineer Pyspark

Job Title

Experience:

Data Engineer (PySpark)

Key Responsibilities:

• Design and define

  data architecture, data strategy, and roadmap

  for large and complex organizations
• Implement

  end-to-end Data Management & Analytics solutions

  at scale
• Transform traditional

  Data Warehousing

  approaches into

  Big Databased architectures

• Manage

  data security, risk mitigation, and governance

  practices
• Design and implement

  dimensional data models

  including Star & Snowflake schemas
• Handle

  Slowly Changing Dimensions (SCDs), role-playing dimensions, and hierarchies

• Drive

  cloud-native designs and deployments

• Enhance and support

  CI/CD pipelines

  for data engineering workloads
• Ensure

  data quality, profiling, governance, metadata management, and archival

• Define and execute

  workload migration strategies

  using appropriate tools
• Collaborate with multiple internal IT teams in a

  matrixed environment

• Work independently on planning, estimation, scheduling, and delivery
  

Mandatory Skills:

• Strong hands-on experience with

  PySpark

  and

  Python

• Expertise in

  Big Data platforms

  : Hadoop, Hive, Spark, Presto, Tez
• Strong understanding of

  Data Warehousing concepts & dimensional modeling

• Experience with

  cloud-native principles, design, and deployments

• Hands-on experience with

  Linux, Git

  , and big data ecosystems
• Experience in

  CI/CD environments

Desired Skills & Competencies:

• Strong problem-solving and analytical skills
  
• Excellent communication, presentation, and influencing abilities
  
• Ability to work in a

  fast-paced, dynamic environment

• Self-starter with strong ownership mindset