Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Data Bricks, Pyspark, SQL and Python and medallion architecture
• Collaborate with cross-functional teams to gather requirements and design solutions for complex data engineering projects.
• Develop scalable architectures for storing and processing massive datasets on cloud platforms like AWS or GCP.
• Troubleshoot issues related to data quality, performance optimization and security in production environments.

Job Requirements :

• 5-10 years of experience as a Data Engineer with expertise in Data Bricks, Pyspark and Python programming languages.
• Strong understanding of SQL concepts and ability to write efficient queries for querying large datasets.
• Experience working with big-data technologies such as Hadoop ecosystem (HDFS) is required.