Senior Data Engineer

Job Description

About The Role We are seeking a highly skilled Senior Data Engineer to join our team and drive the development of our data platform. This role will focus on building scalable data pipelines, optimizing data storage, and ensuring data security and governance. You will collaborate with product teams and other engineering teams to build efficient, high-performance data solutions. Key Responsibilities Design, develop, and optimize scalable ETL/ELT pipelines using Databricks and Apache Spark for both batch and streaming data processing. Develop and maintain data models that support analytical and operational workloads. Collaborate with product teams and other engineering teams to understand data requirements and design robust solutions. Ensure data security, access control, and governance frameworks are implemented and adhered to. Monitor data quality and performance, identifying areas for optimization and improvement. Ensure code quality and implement test cases on data pipeline codes. Follow the latest trends/updates in the data side across the industry and validates its applications on improving our data platform. Requirements Required Skills 5+ years of experience in data engineering with a focus on large-scale data processing. Expertise in Databricks, Apache Spark, and SQL. Strong experience in batch and streaming data processing. Strong experience in Python and PySpark for data processing and automation. Strong experience in data modeling for both Operational and Analytical data. Strong problem-solving skills and ability to optimize performance in distributed data environments. Hands-on experience with data security, access control, and governance frameworks. Experience with Apache Druid/ MSSQL/ MySQL/ MongoDB/ Oracle/ PostgreSQL. Good to have experience with Graph Databases to manage complex relationships within data Good to have experience in working with Open-Source community. Benefits Comprehensive Medical Insurance – Covers the entire family, ensuring health and well-being for employees and their loved ones. Hybrid Work Culture – Only 3 days in-office per week, offering flexibility and reducing commute stress. Healthy Work-Life Balance – Prioritizes employee well-being, allowing time for personal and professional growth. Friendly Working Environment – A supportive and collaborative workplace culture that fosters positivity and teamwork. No Variable Pay Deductions – Guaranteed full compensation with no unpredictable pay cuts, ensuring financial stability. Certification Reimbursement - Company will reimburse the money required to take certificates based on project demands Show more Show less