Aws Sr Data Engineer

Job Title: Data Engineer

ABOUT THE ROLE

PRIMARY RESPONSIBILITIES

Knowledge & Skills Requirement

• Advanced proficiency in Python for data engineering tasks, including experience with libraries such as Pandas, NumPy.
• Expert-level SQL skills with ability to write complex queries, optimize query performance, and design efficient database schemas
• Experience with version control systems (Git) and collaborative development practices Data Integration & ETL/ELT:
• Hands-on experience in building and maintaining ETL/ELT pipelines
• Proficiency with Apache Spark for distributed data processing
• Experience with workflow orchestration tools, particularly Apache Airflow
• Strong understanding of API design and integration patterns (REST, GraphQL)
• Knowledge of message queuing systems (Kafka, RabbitMQ) is a plus.

Data Storage & Management:

• Experience with SQL/NoSQL databases and understanding of when to use each type
• Hands-on experience with data warehousing platforms such as Amazon Redshift or Snowflake
• Knowledge of or experience working with geospatial (GIS) data.
• Understanding of data modeling techniques (dimensional modeling, normalization, denormalization)

Cloud & Distributed Computing:

• Experience with AWS cloud platform and managed data services (S3, Redshift, Glue, Lambda, RDS)
• Experience with distributed computing frameworks and big data technologies
• Understanding of cloud cost optimization and resource management Data Quality & Governance:
• Strong expertise in data cleaning, validation, and quality assurance techniques
• Experience with data profiling tools and methodologies
• Understanding of data governance principles and best practices
• Ability to implement data lineage and metadata management solutions

• Reporting & Visualization:

• Experience working with BI tools, particularly Power BI
• Ability to design data models that support efficient reporting and visualization

Professional Skills

• Strong problem-solving abilities with analytical and critical thinking skills

• Excellent communication skills with ability to explain technical concepts to non-technical stakeholders

• Proven ability to work collaboratively in cross-functional teams

• Experience in client handling, including gathering requirements, managing expectations, and providing regular status updates

Preferred Qualifications

• Bachelors degree in computer science, Information Systems, Data Engineering, or related field

• Experience with real-time data streaming and event-driven architectures

• Certifications in AWS :- o AWS Certified Solutions Architect Associate

o AWS Certified Developer Associate

o AWS Certified Solutions Architect Professional

• Experience with CI/CD practices for data pipelines

• Understanding of data security, encryption, and compliance requirements (GDPR, CCPA)