Area Manager-Data Engineering-Cloud Data Engineering

Role Overview: As a Data Engineer Consultant, you will be responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders. Your primary focus will be on utilizing your strong knowledge of Python, Pyspark, SQL, Hadoop, Hive, Azure, Databricks, and Greenplum to create efficient data workflows and ensure data quality. Key Responsibilities: - Write Pyspark scripts to develop data workflows - Query metadata and tables using SQL from various data management systems like Oracle, Hive, Databricks, and Greenplum - Utilize big data technologies including Hadoop, Spark, and distributed computing frameworks - Use tools like Hue to run Hive SQL queries and schedule Apache Oozie jobs for automating data workflows - Communicate effectively with stakeholders and collaborate with the business team for data testing - Demonstrate strong problem-solving and troubleshooting skills - Establish comprehensive data quality test cases, procedures, and implement automated data validation processes Qualifications Required: - Degree in Data Science, Statistics, Computer Science, or related fields - 5-7 years of experience in Data Engineering - Proficiency in programming languages like Python, Pyspark, and SQL - Experience with Azure cloud computing platform, including ETL processes using Azure Data Factory and big data processing with Azure Databricks - Strong communication, problem-solving, and analytical skills with the ability to manage time effectively and multitask while maintaining attention to detail and accuracy,