Engineer/Senior Engineer - Data Engineering

As a Senior Data Engineer at IndiaMart, you will be responsible for designing, implementing, and optimizing data pipelines, ETL processes, and data warehousing solutions. You will collaborate closely with cross-functional teams including product managers, data scientists, analysts, and software engineers to ensure the availability, accuracy, and efficiency of our data ecosystem. The role requires a deep understanding of data engineering best practices, modern data technologies, and a passion for building scalable and robust data systems.

Responsibilities:

• Design, develop, and maintain efficient and scalable data pipelines to support various data initiatives, ensuring the timely delivery of high-quality data to stakeholders.
• Collaborate with data scientists, analysts, and other teams to understand their data requirements and assist in building data solutions that meet those needs.
• Implement ETL processes for extracting, transforming, and loading data from various sources into data warehouses and other storage systems.
• Optimize and fine-tune existing data pipelines for performance and reliability, identifying and resolving bottlenecks and inefficiencies.
• Work with cloud-based technologies (e.g., AWS, Azure, GCP) to build and manage data infrastructure in a cost-effective and secure manner.
• Monitor data pipelines, troubleshoot issues, and implement proactive solutions to ensure data integrity and availability.
• Contribute to the development of data architecture and governance frameworks to maintain data quality, consistency, and security.
• Mentor and provide technical guidance to junior data engineers, fostering skill development and knowledge sharing within the team.
• Stay up-to-date with emerging trends and advancements in data engineering and technology, and evaluate their potential impact on the company's data ecosystem.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field; Master's degree is a plus.
• Proven experience in data engineering with a focus on designing and implementing complex data pipelines and ETL processes.
• Proficiency in programming languages such as Golang, Python, Java, or similar languages commonly used in data engineering.
• Strong experience with data warehousing solutions (e.g., Redshift, BigQuery) and cloud-based technologies (AWS, Azure, GCP).
• Solid understanding of database technologies, both relational and NoSQL, and experience in query optimization and performance tuning. PostgreSQL knowledge is mandatory.
• Familiarity with data modeling concepts and data architecture best practices.
• Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow, Apache Kafka, Apache Spark, Debezium) and version control systems (e.g., Git).
• Hands-on experience with deployments of pipelines using Kubernetes.
• Excellent problem-solving skills and the ability to diagnose and resolve complex data engineering issues.
• Strong communication skills, with the ability to collaborate effectively with technical and non-technical stakeholders.
• Prior experience in mentoring or leading junior team members is a plus.