Consultant ,Data Engineering

Job Description

About The Role We are seeking a highly experienced and skilled Consultant in Data Engineering to join our team. In this role, you will be responsible for designing, developing, and implementing robust and scalable data engineering solutions for our clients. You will leverage your deep expertise in big data technologies, data warehousing, and cloud platforms to build efficient data pipelines and ensure data quality. Key Responsibilities Design and develop scalable data engineering solutions using Spark and Python. Build and maintain efficient data pipelines for data ingestion, processing, and transformation. Work extensively with large datasets and distributed computing frameworks. Utilize strong coding skills in Python and Spark to implement complex data processing logic. Leverage Greenplum and Teradata for data warehousing and analytical purposes. Develop and maintain Unix shell scripts for automation and system administration tasks. Potentially work with Azure cloud services for data storage and processing. May involve working with Kafka for real-time data streaming. Ensure data quality, integrity, and performance of data pipelines. Troubleshoot and resolve data-related issues. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective solutions. Contribute to the development of data engineering best practices and standards. Stay up-to-date with the latest advancements in data engineering technologies. Qualifications & Skills Bachelor's or Master's degree in Computer Science, Engineering, or a related field. 12-15 years of overall experience in the IT industry. Minimum 12 years of hands-on experience specifically in Data Engineering. Strong proficiency in Python and Spark is mandatory. Significant experience working with Greenplum and Teradata data warehousing platforms. Solid understanding of data engineering principles and best practices. Experience with Unix/Linux operating systems and shell scripting. Exposure to Azure cloud data services and Kafka is a plus. Excellent problem-solving and analytical skills. Strong communication and collaboration abilities. Ability to work independently and as part of a team. To Apply Interested and qualified candidates with extensive experience in Data Engineering, particularly with the mandatory and primary skills mentioned, are encouraged to apply with their updated resume. Skills: python,teradata,kafka,azure,greenplum,cloud,data engineering,data,shell scripting,spark,unix/linux Show more Show less