Data Engineer Big Data

As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable and efficient big data processing pipelines in distributed computing systems. You will collaborate with cross-functional teams to understand data requirements and design appropriate data solutions. Your role will involve implementing data ingestion, processing, and transformation processes to support various analytical and machine learning use cases. In this position, you will optimize and tune data pipelines to enhance performance, scalability, and reliability. You will also be responsible for monitoring and troubleshooting pipeline performance issues, identifying and resolving bottlenecks. Ensuring data quality and integrity throughout the pipeline by implementing data validation and error handling mechanisms is a key aspect of your responsibilities. As a Data Engineer, it is important to stay updated on emerging technologies and best practices in big data processing and analytics. You will be expected to incorporate these new developments into our data engineering practices. Additionally, documenting design decisions, technical specifications, and data workflows will be part of your regular tasks to maintain transparency and facilitate knowledge sharing within the team.,