Location: Hyderabad, Noida, Gurugram
About the Role
We are seeking a highly experienced and passionate Data Engineer to join our growing data team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining our robust data infrastructure, ensuring data quality, availability, and scalability. You will play a critical role in empowering our data-driven decision-making by building and optimizing data pipelines, ETL processes, and data warehousing solutions. This is a senior-level position requiring extensive experience with database technologies, data modelling, and cloud platforms.
Responsibilities:
-
Data Pipeline Development & Maintenance:
Design, build, and maintain end-to-end data pipelines for ingesting, processing, and transforming large datasets from various sources (e.g., relational databases, APIs, flat files, streaming data). -
ETL/ELT Process Optimization:
Develop and optimize ETL/ELT processes using industry-standard tools and techniques, ensuring data accuracy, efficiency, and scalability. -
Database Design & Management:
Design, implement, and manage relational databases, including schema design, indexing, performance tuning, and data governance. -
Data Warehousing:
Design and implement data warehousing solutions to support business intelligence and reporting needs. Experience with star schema, snowflake schema, and other data modelling techniques is essential. -
Database Administration:
Perform database administration tasks, including performance monitoring, capacity planning, backup and recovery, and security management. -
Data Quality & Governance:
Implement data quality checks, validation rules, and data governance policies to ensure data accuracy and consistency. -
Cloud Platform Expertise:
Leverage cloud platforms (e.g., AWS, Azure, GCP) for data storage, processing, and management. -
Collaboration & Communication:
Collaborate with cross-functional teams (e.g., data scientists, business analysts, software engineers) to understand data requirements and deliver effective data solutions. Clearly communicate technical concepts to both technical and non-technical audiences. -
Mentoring & Knowledge Sharing:
Mentor junior engineers and share knowledge and best practices within the team. -
Automation & Scripting:
Automate data engineering tasks using scripting languages (e.g., Python, Bash). -
Stay Up to Date:
Continuously research and evaluate new data technologies and techniques to improve our data infrastructure. -
Production Support:
Monitoring the batch/jobs daily, no matter it s weekdays or weekend. -
Production Release:
Actively participation in release process.
Qualifications:
-
Experience:
Minimum of 10 years of experience as a Data Engineer or in a similar role. -
Database Expertise:
-
Expert proficiency in SQL Server:
Extensive experience with SQL Server, including database design, performance tuning, query optimization, and database administration. -
Expert proficiency in PostgreSQL:
Deep understanding of PostgreSQL, including database design, performance tuning, query optimization, and database administration. - Solid experience with other relational databases like MySQL, Oracle, etc. is a plus.
-
ETL/ELT Tools:
Proven experience with ETL/ELT tools (e.g., Apache Airflow, Informatica, Talend, SSIS, ADF etc.). -
Data Modelling:
Strong understanding of data modelling principles and techniques (e.g., dimensional modelling, star schema, snowflake schema). -
Cloud Computing:
Experience with cloud platforms (AWS, Azure, or GCP) and related data services (e.g., S3, Redshift, Snowflake, Azure Data Lake Storage). -
Programming & Scripting:
Proficiency in scripting languages such as Python, Bash, or similar. -
Data Governance & Quality:
Experience implementing data quality checks, data governance policies, and data validation rules. -
Problem-Solving & Analytical Skills:
Excellent problem-solving and analytical skills with the ability to identify and resolve complex data-related issues. -
Communication & Collaboration:
Excellent communication, collaboration, and interpersonal skills. Ability to work effectively in a team environment. -
Education:
Bachelor s or master s degree in computer science, Information Technology, or a related field.
Bonus Points:
- Experience with data streaming technologies (e.g., Kafka, Spark Streaming).
- Experience with data visualization tools (e.g., Tableau, Power BI, Looker).
- Relevant certifications (e.g., AWS Certified Data Engineer, Microsoft Certified: Azure Data Engineer Associate).
