Data Engineer (AWS)

Note:

Roles & Responsibilities

• Design, implement, and improve the analytics platform.
• Implement and simplify self-service data query and analysis capabilities of the BI platform.
• Develop and improve the current BI architecture, emphasizing data security, data quality, timeliness, scalability, and extensibility.
• Deploy and use various big data technologies and run pilots to design low-latency data architectures at scale.
• Collaborate with business analysts, data scientists, product managers, software development engineers, and other BI teams to develop, implement, and validate KPIs, statistical analyses, data profiling, prediction, forecasting, clustering, and machine learning algorithms.

Educational Qualifications

Experience:

• Minimum 3 years of hands-on experience in data engineering, with a proven track record in complex pipeline development and cloud-based data migration projects.

Education:

• Bachelors or higher degree in Computer Science, Data Engineering, or a related field.

Skills:

• Proficiency in Spark, SQL, Python, and other relevant data processing technologies.
• Expertise in on-premises to cloud Spark code optimization and Medallion Architecture.
• Strong knowledge of Databricks and its components, including Delta Live Table (DLT) pipeline implementations.
• Familiarity with AWS services (experience with additional cloud platforms like GCP or Azure is a plus).

Soft Skills:

• Excellent communication and collaboration skills, with the ability to work effectively with clients and internal teams.

Good to Have

Certifications:

• AWS/GCP/Azure Data Engineer Certification.